-- Logs begin at Thu 2022-05-26 20:10:18 UTC, end at Thu 2022-05-26 21:29:37 UTC. -- May 26 20:49:28.924892 dv-cinder-828806-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 26 20:49:28.933544 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 26 20:49:28.935534 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 26 20:49:28 2022] *** May 26 20:49:28.935534 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 26 20:49:28.935534 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 26 20:49:28.935534 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: nodename: dv-cinder-828806-3-iscsi May 26 20:49:28.935534 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: machine: x86_64 May 26 20:49:28.935534 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: clock source: unix May 26 20:49:28.935534 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: pcre jit disabled May 26 20:49:28.935534 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: detected number of CPU cores: 4 May 26 20:49:28.935534 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: current working directory: / May 26 20:49:28.935534 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: detected binary path: /usr/bin/uwsgi-core May 26 20:49:28.936068 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: your processes number limit is 47680 May 26 20:49:28.936068 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: your memory page size is 4096 bytes May 26 20:49:28.936068 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: detected max file descriptor number: 2048 May 26 20:49:28.936068 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: lock engine: pthread robust mutexes May 26 20:49:28.936068 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: thunder lock: enabled May 26 20:49:28.936068 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 26 20:49:28.936068 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 26 20:49:28.980715 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: Python main interpreter initialized at 0x556a6a64a9c0 May 26 20:49:28.980715 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: python threads support enabled May 26 20:49:28.980715 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: your server socket listen backlog is limited to 100 connections May 26 20:49:28.980715 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: your mercy for graceful operations on workers is 90 seconds May 26 20:49:28.981026 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: mapped 403077 bytes (393 KB) for 2 cores May 26 20:49:28.981121 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: *** Operational MODE: preforking *** May 26 20:49:28.981393 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: *** uWSGI is running in multiple interpreter mode *** May 26 20:49:28.981393 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: spawned uWSGI master process (pid: 80259) May 26 20:49:28.981578 dv-cinder-828806-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 26 20:49:28.982939 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: spawned uWSGI worker 1 (pid: 80268, cores: 1) May 26 20:49:28.982939 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: spawned uWSGI worker 2 (pid: 80269, cores: 1) May 26 20:49:28.982939 dv-cinder-828806-3-iscsi devstack@keystone.service[80259]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 26 20:49:34.604969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 20:49:34.604969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: from cryptography.utils import int_from_bytes May 26 20:49:34.606360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 20:49:34.606360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: from cryptography.utils import int_from_bytes May 26 20:49:34.643030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 20:49:34.643030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: from cryptography.utils import int_from_bytes May 26 20:49:34.643511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 20:49:34.643511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: from cryptography.utils import int_from_bytes May 26 20:49:34.655450 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 20:49:34.655646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 20:49:34.655838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] command line args: [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 20:49:34.656108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 20:49:34.656286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ================================================================================ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 20:49:34.656563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] admin_token = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.656813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_dir = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.657003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.657184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] config_source = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.657365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] debug = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.657572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.658998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.658998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.658998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.658998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.658998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.658998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_config_append = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.659647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.659647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_dir = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.659647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.659899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.660020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.660197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.660446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.660625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.660808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.661053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.661300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.661535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.661768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.662000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.662232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.662498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.662746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.663022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.663507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.663694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] publish_errors = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.663933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.664174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.664354 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.664606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.664843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.665080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] standard_threads = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.665313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.665558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.665735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.665913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_journal = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.666089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_json = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.666263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_stderr = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.666438 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] use_syslog = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.666694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.666952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.667193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.667453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.667693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.667942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.668737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.668990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.669229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.669478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.669727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.669978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.670215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.670480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.670724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.670974 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.671223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.671460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.671716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.671970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.672218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.672456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.672712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.672974 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.673229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.673481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.673717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.673950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.674189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.674427 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.674702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.674954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.675186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.675419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.675667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.675915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.676154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.676405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.676639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.676876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.677111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.677343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.677575 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.677806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.678037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.678274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.678547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.678781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.679017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.679249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.679485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.679721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.679971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.680203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.680441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.680677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.680914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.681151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.681388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.681621 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.681871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.682108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.682347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.682602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.682864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.683103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.683336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.683572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.684329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.684598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.684840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.685089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.685337 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.685575 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.685812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.686049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.686299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.686571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.686813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.687057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.687293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.687529 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.687766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.688810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.689055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.689322 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.689564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.689819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.690060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.690305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.690569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.690810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.691049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.691282 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.691516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.691748 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.691985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.692217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.692459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.692696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.692945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.693183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.693442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.693705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.693787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 20:49:34.693934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 20:49:34.694020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.694131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] command line args: [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 20:49:34.694214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.external = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.694386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.694486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 20:49:34.694622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ================================================================================ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 20:49:34.694778 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.694886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] admin_token = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.694962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.695143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config_dir = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.695219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.password = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.695338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.695422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] auth.token = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.695526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] config_source = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.695603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.695713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] debug = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.695825 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.695931 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.696073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.696175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.696306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.696383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.696554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.696633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.696801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.696878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.697034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.697110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.697245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_config_append = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.697327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.697428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.697504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.697611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_dir = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.697742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.697827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.697991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.698079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.698229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.698305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.698493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.698584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.698734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.698808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.698937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.699015 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.699120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.699237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.699912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.700027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.700180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.700293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.700443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.700570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.700656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.700821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.700906 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.701088 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.701180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.701332 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.701408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.701581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.701671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.701818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.701905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.702069 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.702303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.702384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.702543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] publish_errors = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.702623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.702783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.702870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.703020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.703104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.703201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.703284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.703456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.703534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.703697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.703774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.703938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] standard_threads = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.704024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.704175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.704257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.704427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.704512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.704607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.704687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.704789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_journal = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.704916 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.704993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_json = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.705133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_stderr = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.705208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.705313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] use_syslog = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.705390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.705549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 20:49:34.705625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.705810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.705886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.706054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.706139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.706299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.706376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.706578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.706659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.706833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.706911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.707068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.707146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.707302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.707379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.707523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.707643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.707783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.707867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.708035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.708111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.708290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.708368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.708532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.708608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.708786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.708895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.709031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.709177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.709286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.709420 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.709540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.709660 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.709782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.709895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.710051 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.710128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.710312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.710390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.710588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.710676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.710832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.710909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.711070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.711152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.711320 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.711405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.711552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.711629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.711800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.711877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.712049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.712134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.712285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.712361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.712523 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.712600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.712765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.712849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.712993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.713070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.713231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.713308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.713462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.713566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.713697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.713782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.713934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.714019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.714174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.714258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.714411 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.714529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.714673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.714751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.714947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.715023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.715196 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.715288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.715426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.715547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.715659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.715738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.715901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.715985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.716140 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.716218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.716375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.716452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.716616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.716700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.716851 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.716927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.717084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.717161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.717324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.717411 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.717561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.717638 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.717800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.717883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.718039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.718123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.718284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.718361 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.718541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.718624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.718789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.718878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.719018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.719106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.719255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.719334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.719490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.719576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.719725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.719803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.719956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.720033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.720208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.720295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.720451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.720528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.720695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.720772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.720934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.721019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.721167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.721240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.721393 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.721477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.721640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.721726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.721877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.721955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.722114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.722202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.722350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.722437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.722611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.722689 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.722857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.722935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.723105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.723194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.723341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.723419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.723590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.723675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.723818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.723903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.724062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.724139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.724320 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.724452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.724579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.724719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.724844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.724967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.725094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.725219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.725355 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.725488 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.725612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.725738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.725861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.725980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.726109 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.726242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.726242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.726496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.726619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.726741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.726873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.727001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.727135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.727257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.727379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.727500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.727630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.727754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.727882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.728004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.728127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.728248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.728378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.728500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.728622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.728743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.728876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.728997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.729127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.729249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.729371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.729491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.729613 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.729734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.729865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.729988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.730110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.730235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.730357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.730543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.730677 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.730804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.730930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.731054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.731180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.731304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.731435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.external = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.731558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.731683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.731807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.731931 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.732058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.732192 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.732343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.732469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.password = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.732593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.732717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] auth.token = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.732841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.732973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.733099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.733225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.733349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.733474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.733598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.733728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.733853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.733987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.734113 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.734237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.734362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.734514 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.734637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.734759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.734880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.735002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.735124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.735255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.735383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.735506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.735636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.735758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.735888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.736010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.736133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.736256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.736378 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.736500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.736629 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.736751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.736873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.736994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.737116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.737239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.737378 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.737500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.737622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.737744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.737865 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.737987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.738116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.738239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.738362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.738362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.738622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.738748 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.738880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.739005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.739137 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.739261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.739384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.739507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.739637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.739760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.739883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.740005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.740133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.740257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.740390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.740514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.740637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.740773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.740898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.741020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.741152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.741277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.741406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.741530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.741652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.741774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.741904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.742026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.742159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.742291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.742415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.742564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.742687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.742884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.743008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.743131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.743253 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.743376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.743498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.743629 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.743752 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.743873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.743995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.744116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.744247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.744377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.744500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.744622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.744744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.744865 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.744993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.745123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.745246 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.745368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.745490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.745618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.745740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.745869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.746001 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.746124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.746776 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.746903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.747028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.747162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.747162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80269) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 20:49:34.747398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.747398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.747398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.747398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.747398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.747398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.747398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.747398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.748308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.748308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.748308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.748308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.748770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.748770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.749003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.749125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.749369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.749607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.749850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.750097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.750342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.750609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.750855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.751113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.751368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.751607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.751844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.752099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.752354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.752595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.752836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.753077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.753312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.753557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.753821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.754118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.754397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.754725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.755045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.755446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.755781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.756037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.756280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.756525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.756865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.757127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.757387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.757645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.757902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.758163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.758404 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.758687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.758934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.759179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.759425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.759670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.759928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.760185 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.760430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.760694 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.760933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.761184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.761428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.761672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.762492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.762754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.763260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.763510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.763756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.764033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.764293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.764551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.764797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.765056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.765314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.765564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.765819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.766065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.766313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.766584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.766842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.767081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.767334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.767592 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.767848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.768103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.768360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.768616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.768871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.769125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.769364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.769617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.769872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.770134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.770381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.770816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.771060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.771318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.771574 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.771829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.772096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.772338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.772599 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.772858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.773103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.773341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.773580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.773817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.774074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.774334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.774614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.774860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.775104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.775342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.775585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.775830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.776086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.776329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.776593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.776850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.777096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.777221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.777358 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.777507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.777633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.777761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.777885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.778104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.778352 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.778617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.778872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.779127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.779375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.779624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.779929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.780199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.780448 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.780702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.780946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.781193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.781436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.781680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.781922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.782164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.782415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.782684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.782935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.783177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.783433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.783676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.783926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.784164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.784411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.784654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 20:49:34.784830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80268) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 20:49:34.786035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.809998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.810234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.810481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.814764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.814994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.821178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.840467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.840701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.870622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.878191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.921841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.922068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.925973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.926214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.987100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.987328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.987549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.987767 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.987869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.987982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.988067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.988195 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.988278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.988474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.988650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:34.988870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.011512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.011755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.011856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.011989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.012073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.012211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.012294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.012433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.012524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.012645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.012727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.012857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.012938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.013336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.024295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.024726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.036803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.037050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.037252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.037340 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.037473 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.037555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.037688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.037779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.037910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.037993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.038098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80269) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.038183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.038350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.038594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80268) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 20:49:35.056444 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.056657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 20:49:35.056746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.056933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 20:49:35.057034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.057187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 20:49:35.057267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.057449 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 20:49:35.057529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.057709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.057798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.057968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.058048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.058223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.058303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.058502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.069948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.070229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.070506 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.070786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.071522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.071803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.072082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.072363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.078836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.080468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.082074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.082352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.082655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.082934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.083210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.083465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.083764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.083844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.084045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.084126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.084330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.084418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.084611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.084689 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.084878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.084956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.085254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.085530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.085809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.086086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.086362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.086661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.111399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.113144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.113362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.113443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.115104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.115388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.118689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.118967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.119256 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.120682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.120928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.121801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.127081 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.127362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.129075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.129360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.131844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.132239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.132521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.132815 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.133091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.133385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.133653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.133820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.133929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.134220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.134604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.134895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.135173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.135446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.135717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.135987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.152687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.152959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.154732 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.155012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.157591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.159615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.161218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.161509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.163213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.163513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.166640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.168996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.170743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.172731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.174476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.174771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.175054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.175342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.175624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.175907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.176501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.176802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.177098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.177386 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.177683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.177976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.195153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.195436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.195701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.195962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.197362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.197646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.197914 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.198179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.203886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.205576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.205850 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.206190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.207904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.208200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.208878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.209159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.209441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.209730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.210005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.211261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.211545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.211828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.212109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.212395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.223252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.223532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.223811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.224095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.224374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.224647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.224929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.225751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.226033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.226315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.226626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.226913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.227198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.227474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.243182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.245886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.246977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.249744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.250262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.251867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.253063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.253475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.253761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.254043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.254705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.256323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.256610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.256902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.261346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.263248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.264594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.266202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.266504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.266792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.267073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.267346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.269442 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.269734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.270023 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.270306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.276369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.276655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.277033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.279384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.279674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.280063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.284870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80269) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.285147 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.285428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.285701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.285976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.286251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.286558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.286824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.287096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.287361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.287631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.287902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.288049 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80268) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 20:49:35.288163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.288246 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.288418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80269) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.288546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.288724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.289001 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.289274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.289551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.289838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.290109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.290395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.290690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.290954 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.291228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.291497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80268) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 20:49:35.327057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x556a6a64a9c0 pid: 80269 (default app) May 26 20:49:35.330245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de86ff-56ac-4e2a-b231-9b3cfe417a56 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:35.330393 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x556a6a64a9c0 pid: 80268 (default app) May 26 20:49:35.330562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de86ff-56ac-4e2a-b231-9b3cfe417a56 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:35.330840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de86ff-56ac-4e2a-b231-9b3cfe417a56 None None] PATH_INFO: `/v3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:35.335163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Thu May 26 20:49:35 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:43.208716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caec9f-8d47-4ff1-9f36-3dfe52175fdd None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:43.209237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caec9f-8d47-4ff1-9f36-3dfe52175fdd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:43.209237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caec9f-8d47-4ff1-9f36-3dfe52175fdd None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:43.213627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:49:43.218705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:43.219120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:43.219264 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:43.286740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: INFO dbcounter [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] Registered counter for database keystone May 26 20:49:43.307902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG oslo_db.sqlalchemy.engines [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80269) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 20:49:43.314685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writer thread running {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 20:49:43.487593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG passlib.handlers.bcrypt [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80269) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 20:49:43.487955 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG passlib.handlers.bcrypt [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80269) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 20:49:43.544415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:43.546152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:43.669419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 20:49:43.826860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97caec9f-8d47-4ff1-9f36-3dfe52175fdd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:43.897103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: INFO dbcounter [None req-97caec9f-8d47-4ff1-9f36-3dfe52175fdd None None] Registered counter for database keystone May 26 20:49:43.918644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG oslo_db.sqlalchemy.engines [None req-97caec9f-8d47-4ff1-9f36-3dfe52175fdd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80268) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 20:49:43.924786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writer thread running {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 20:49:44.045233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97caec9f-8d47-4ff1-9f36-3dfe52175fdd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:44.049527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503df70-f30c-4f4d-a390-a939581a3da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:44.050189 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503df70-f30c-4f4d-a390-a939581a3da9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:44.050505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503df70-f30c-4f4d-a390-a939581a3da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:44.050797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503df70-f30c-4f4d-a390-a939581a3da9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:44.059730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503df70-f30c-4f4d-a390-a939581a3da9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:44.110522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5503df70-f30c-4f4d-a390-a939581a3da9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 20:49:44.110830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503df70-f30c-4f4d-a390-a939581a3da9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:44.120359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 26 20:49:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:49:44.127017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:44.141460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5fa75-61e6-45e9-9330-8b261d7d31c3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:44.145744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584d534-3cc5-45b0-8c47-c490473a2ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:44.146891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584d534-3cc5-45b0-8c47-c490473a2ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:44.147179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584d534-3cc5-45b0-8c47-c490473a2ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:44.147455 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584d534-3cc5-45b0-8c47-c490473a2ed7 admin admin] PATH_INFO: `/v3/projects/116e4cb137e24acda75f964e89e322c5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:44.148947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584d534-3cc5-45b0-8c47-c490473a2ed7 admin admin] RBAC: Authorizing `identity:get_project(project_id=116e4cb137e24acda75f964e89e322c5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:44.320768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c584d534-3cc5-45b0-8c47-c490473a2ed7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 20:49:44.321412 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584d534-3cc5-45b0-8c47-c490473a2ed7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:44.325833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Thu May 26 20:49:44 2022] GET /identity/v3/projects/116e4cb137e24acda75f964e89e322c5 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:46.507420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0ac4a-946e-4885-988e-aef40a535fe2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:46.508167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0ac4a-946e-4885-988e-aef40a535fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:46.508167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0ac4a-946e-4885-988e-aef40a535fe2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:46.514125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:49:46.519800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed3583-23d7-485a-bf63-32adc7c30ee7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:46.520222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed3583-23d7-485a-bf63-32adc7c30ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:46.520362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed3583-23d7-485a-bf63-32adc7c30ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:46.539857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ceed3583-23d7-485a-bf63-32adc7c30ee7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:46.541429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ceed3583-23d7-485a-bf63-32adc7c30ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:46.549298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 20:49:46.571549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0ac4a-946e-4885-988e-aef40a535fe2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:46.584370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0ac4a-946e-4885-988e-aef40a535fe2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:46.588583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d404b5-ec8b-4f0e-b077-46ef87c8d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:46.589218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d404b5-ec8b-4f0e-b077-46ef87c8d2ac admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:46.589504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d404b5-ec8b-4f0e-b077-46ef87c8d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:46.589826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d404b5-ec8b-4f0e-b077-46ef87c8d2ac admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:46.597462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d404b5-ec8b-4f0e-b077-46ef87c8d2ac admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:46.599264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d404b5-ec8b-4f0e-b077-46ef87c8d2ac admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:46.607936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Thu May 26 20:49:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 20:49:46.613351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed3583-23d7-485a-bf63-32adc7c30ee7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:46.622848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed3583-23d7-485a-bf63-32adc7c30ee7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:46.627184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e677098-c886-4982-8886-f931fad62b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:46.628032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677098-c886-4982-8886-f931fad62b2a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:46.628314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677098-c886-4982-8886-f931fad62b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:46.628587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677098-c886-4982-8886-f931fad62b2a admin admin] PATH_INFO: `/v3/users/c2ba123046b04572883d482380082522` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:46.629959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e677098-c886-4982-8886-f931fad62b2a admin admin] RBAC: Authorizing `identity:get_user(user_id=c2ba123046b04572883d482380082522)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:46.631814 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e677098-c886-4982-8886-f931fad62b2a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:46.633367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Thu May 26 20:49:46 2022] GET /identity/v3/users/c2ba123046b04572883d482380082522 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:48.890562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c882e-7de4-4a9c-9ab9-d21ad813c936 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:48.891061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c882e-7de4-4a9c-9ab9-d21ad813c936 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:48.891159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c882e-7de4-4a9c-9ab9-d21ad813c936 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:48.892830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:49:48.894373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130131b4-2732-4ac6-afaa-61e31d459aaf None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:48.894777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130131b4-2732-4ac6-afaa-61e31d459aaf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:48.895086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130131b4-2732-4ac6-afaa-61e31d459aaf None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:48.896490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:49:48.897633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821b42-7e9d-463c-8de9-11f5e5067a25 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:48.897932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821b42-7e9d-463c-8de9-11f5e5067a25 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:48.898218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33821b42-7e9d-463c-8de9-11f5e5067a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:48.901283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94aed7-3a4c-449f-a3ba-a98e10871d0f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:48.901572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94aed7-3a4c-449f-a3ba-a98e10871d0f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:48.901856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa94aed7-3a4c-449f-a3ba-a98e10871d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:48.917972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fa94aed7-3a4c-449f-a3ba-a98e10871d0f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:48.919165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fa94aed7-3a4c-449f-a3ba-a98e10871d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:48.938090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG passlib.handlers.bcrypt [None req-33821b42-7e9d-463c-8de9-11f5e5067a25 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80268) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 20:49:48.938562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG passlib.handlers.bcrypt [None req-33821b42-7e9d-463c-8de9-11f5e5067a25 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80268) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 20:49:48.984860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-33821b42-7e9d-463c-8de9-11f5e5067a25 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:48.986470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-33821b42-7e9d-463c-8de9-11f5e5067a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:48.996911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 7/12] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:49:49.020263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94aed7-3a4c-449f-a3ba-a98e10871d0f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:49.043516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:49.085620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 6/13] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:49:49.107127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33821b42-7e9d-463c-8de9-11f5e5067a25 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:49.122536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa94aed7-3a4c-449f-a3ba-a98e10871d0f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:49.155721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdd633-ba0c-42e3-ba9a-7bbbc3900ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:49.156518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdd633-ba0c-42e3-ba9a-7bbbc3900ecb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:49.156800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdd633-ba0c-42e3-ba9a-7bbbc3900ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:49.157083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bdd633-ba0c-42e3-ba9a-7bbbc3900ecb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:49.168248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bdd633-ba0c-42e3-ba9a-7bbbc3900ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-32bdd633-ba0c-42e3-ba9a-7bbbc3900ecb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:49:49.179551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:49:49.180715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:49:49.181680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 8/14] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:49:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:49:49.187307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdd633-ba0c-42e3-ba9a-7bbbc3900ecb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:49.211139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33821b42-7e9d-463c-8de9-11f5e5067a25 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:49.213404 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:49.249083 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287047d-d41e-4a42-aa3a-39aee2f316c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:49.249978 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287047d-d41e-4a42-aa3a-39aee2f316c4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:49.250268 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287047d-d41e-4a42-aa3a-39aee2f316c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:49.250579 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287047d-d41e-4a42-aa3a-39aee2f316c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:49.267076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:49.284793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bdd633-ba0c-42e3-ba9a-7bbbc3900ecb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:49.317402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287047d-d41e-4a42-aa3a-39aee2f316c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:49.317625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4247e-9d05-42f0-b760-0c9f3c5fdf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:49.318099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4247e-9d05-42f0-b760-0c9f3c5fdf39 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:49.318380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4247e-9d05-42f0-b760-0c9f3c5fdf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:49.318685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4247e-9d05-42f0-b760-0c9f3c5fdf39 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:49.318957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287047d-d41e-4a42-aa3a-39aee2f316c4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:49.319429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4247e-9d05-42f0-b760-0c9f3c5fdf39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:49:49.319734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4247e-9d05-42f0-b760-0c9f3c5fdf39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:49.321116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4247e-9d05-42f0-b760-0c9f3c5fdf39 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:49.332502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 9/15] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:49:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:49:49.338118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4247e-9d05-42f0-b760-0c9f3c5fdf39 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:49.415738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 7/16] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:49:49.421675 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287047d-d41e-4a42-aa3a-39aee2f316c4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:49.424702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:49.435572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4247e-9d05-42f0-b760-0c9f3c5fdf39 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:49.468093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a69b7f-5c5e-440f-84a0-3b19b5eb5f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:49.468958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a69b7f-5c5e-440f-84a0-3b19b5eb5f4f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:49.469246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a69b7f-5c5e-440f-84a0-3b19b5eb5f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:49.469524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a69b7f-5c5e-440f-84a0-3b19b5eb5f4f None admin] PATH_INFO: `/v3/users/c2ba123046b04572883d482380082522` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:49.470787 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a69b7f-5c5e-440f-84a0-3b19b5eb5f4f None admin] RBAC: Authorizing `identity:get_user(user_id=c2ba123046b04572883d482380082522)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:49.472257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a69b7f-5c5e-440f-84a0-3b19b5eb5f4f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:49.473867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 10/17] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:49:49 2022] GET /identity/v3/users/c2ba123046b04572883d482380082522 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:49:49.479085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a69b7f-5c5e-440f-84a0-3b19b5eb5f4f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:49.493304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:49.519355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287047d-d41e-4a42-aa3a-39aee2f316c4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:49.574361 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f8724-e97f-4b25-b091-1d65c38443ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:49.575264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f8724-e97f-4b25-b091-1d65c38443ed None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:49.575567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f8724-e97f-4b25-b091-1d65c38443ed None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:49.575855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f8724-e97f-4b25-b091-1d65c38443ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:49.582724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a69b7f-5c5e-440f-84a0-3b19b5eb5f4f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:49.594870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f8724-e97f-4b25-b091-1d65c38443ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:49.596296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f8724-e97f-4b25-b091-1d65c38443ed None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:49.616784 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8d1c2-0ce6-4ee0-a2e0-3cc204fd6da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-587f8724-e97f-4b25-b091-1d65c38443ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:49:49.617434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:49:49.618329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:49:49.619358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:49:49.619358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:49:49.619358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:49:49.619358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:49:49.619614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8d1c2-0ce6-4ee0-a2e0-3cc204fd6da2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:49.619614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8d1c2-0ce6-4ee0-a2e0-3cc204fd6da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:49.619614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8d1c2-0ce6-4ee0-a2e0-3cc204fd6da2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:49.619828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 8/18] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:49:49.620894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8d1c2-0ce6-4ee0-a2e0-3cc204fd6da2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:49.622359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8d1c2-0ce6-4ee0-a2e0-3cc204fd6da2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:49.623981 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 11/19] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:49:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:49:49.625820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f8724-e97f-4b25-b091-1d65c38443ed None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:49.629361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8d1c2-0ce6-4ee0-a2e0-3cc204fd6da2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:49.681204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:49.707697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:49.724540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f8724-e97f-4b25-b091-1d65c38443ed None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:49.727696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8d1c2-0ce6-4ee0-a2e0-3cc204fd6da2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:49.757276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b815c93-16be-4b52-8125-7e39c3509602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:49.757932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b815c93-16be-4b52-8125-7e39c3509602 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:49.758220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b815c93-16be-4b52-8125-7e39c3509602 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:49.758526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b815c93-16be-4b52-8125-7e39c3509602 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:49.759249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b815c93-16be-4b52-8125-7e39c3509602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:49:49.759559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b815c93-16be-4b52-8125-7e39c3509602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:49.760955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b815c93-16be-4b52-8125-7e39c3509602 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:49.763875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77868a-0aa3-462a-adae-07c0ff7c717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:49.764517 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77868a-0aa3-462a-adae-07c0ff7c717a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:49.764807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77868a-0aa3-462a-adae-07c0ff7c717a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:49.765092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77868a-0aa3-462a-adae-07c0ff7c717a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:49.765924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77868a-0aa3-462a-adae-07c0ff7c717a None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.domain.id=default, user.id=c2ba123046b04572883d482380082522) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:49:49.766242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77868a-0aa3-462a-adae-07c0ff7c717a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:49.767723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77868a-0aa3-462a-adae-07c0ff7c717a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:49.774364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 12/20] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 26 20:49:49 2022] GET /identity/v3/role_assignments?user.id=c2ba123046b04572883d482380082522&scope.domain.id=default&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:49.778823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 9/21] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:52.298756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe12c1-44a6-4482-aafe-b2f6ae63d02c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:52.299252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe12c1-44a6-4482-aafe-b2f6ae63d02c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:52.299351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe12c1-44a6-4482-aafe-b2f6ae63d02c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:52.300693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 13/22] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:49:52.316422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59957d02-8c37-4646-af32-f64540804bb8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:52.317533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59957d02-8c37-4646-af32-f64540804bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:52.318027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59957d02-8c37-4646-af32-f64540804bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:52.344357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-59957d02-8c37-4646-af32-f64540804bb8 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:52.345881 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-59957d02-8c37-4646-af32-f64540804bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:52.488929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ce465-9c94-4696-84dc-4bbed2abd444 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:52.489599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ce465-9c94-4696-84dc-4bbed2abd444 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:52.490093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06ce465-9c94-4696-84dc-4bbed2abd444 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:52.492139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 14/23] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:49:52.498364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91658d1c-785d-45a5-ada7-6b0d8646d8f9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:52.499011 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91658d1c-785d-45a5-ada7-6b0d8646d8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:52.499504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91658d1c-785d-45a5-ada7-6b0d8646d8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:52.539484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-91658d1c-785d-45a5-ada7-6b0d8646d8f9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:52.541078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-91658d1c-785d-45a5-ada7-6b0d8646d8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:52.554012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 10/24] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:49:52.598254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59957d02-8c37-4646-af32-f64540804bb8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:52.657943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 15/25] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:49:52.660692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481d716-0ffb-458d-83b4-05f8c583c080 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:52.661874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481d716-0ffb-458d-83b4-05f8c583c080 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:52.662988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481d716-0ffb-458d-83b4-05f8c583c080 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:52.664620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 16/26] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:49:52.679885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054829e1-0ca1-4cb0-a49c-00858aa19d4e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:52.680467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054829e1-0ca1-4cb0-a49c-00858aa19d4e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:52.680967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054829e1-0ca1-4cb0-a49c-00858aa19d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:52.700302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:52.707512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-054829e1-0ca1-4cb0-a49c-00858aa19d4e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:52.708912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-054829e1-0ca1-4cb0-a49c-00858aa19d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:52.806579 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59957d02-8c37-4646-af32-f64540804bb8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:52.835827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 17/27] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:49:52.836681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054829e1-0ca1-4cb0-a49c-00858aa19d4e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:52.846776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:52.854287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe2aea-284e-4a94-9be4-214e7f642344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:52.855165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe2aea-284e-4a94-9be4-214e7f642344 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:52.855453 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe2aea-284e-4a94-9be4-214e7f642344 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:52.855735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe2aea-284e-4a94-9be4-214e7f642344 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:52.875638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe2aea-284e-4a94-9be4-214e7f642344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-54fe2aea-284e-4a94-9be4-214e7f642344 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:49:52.890838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:49:52.891750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:49:52.892505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 11/28] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:49:52.897180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe2aea-284e-4a94-9be4-214e7f642344 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:52.938969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054829e1-0ca1-4cb0-a49c-00858aa19d4e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:52.991187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593517c7-d3d0-4794-a3da-e400dc226dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:52.992179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593517c7-d3d0-4794-a3da-e400dc226dd4 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:52.992639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593517c7-d3d0-4794-a3da-e400dc226dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:52.993059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593517c7-d3d0-4794-a3da-e400dc226dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:52.993953 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593517c7-d3d0-4794-a3da-e400dc226dd4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:52.996091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593517c7-d3d0-4794-a3da-e400dc226dd4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:53.066199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe2aea-284e-4a94-9be4-214e7f642344 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:53.074330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:53.125854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666ee02-50fd-41b2-9350-ef7c148b9b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:53.126979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ee02-50fd-41b2-9350-ef7c148b9b8f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:53.127443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ee02-50fd-41b2-9350-ef7c148b9b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:53.127894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ee02-50fd-41b2-9350-ef7c148b9b8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:53.139731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666ee02-50fd-41b2-9350-ef7c148b9b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:53.141433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666ee02-50fd-41b2-9350-ef7c148b9b8f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:53.157319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 18/29] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 26 20:49:52 2022] POST /identity/v3/roles => generated 219 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:49:53.158308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593517c7-d3d0-4794-a3da-e400dc226dd4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1666ee02-50fd-41b2-9350-ef7c148b9b8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:49:53.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:49:53.168222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:49:53.169025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:49:53.169025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:49:53.169025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:49:53.169025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:49:53.173388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 12/30] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:49:53.180894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666ee02-50fd-41b2-9350-ef7c148b9b8f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:53.344144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:53.424065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593517c7-d3d0-4794-a3da-e400dc226dd4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:53.427697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666ee02-50fd-41b2-9350-ef7c148b9b8f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:53.491744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa7483-256b-48b9-a2e9-b971b49afc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:53.492429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa7483-256b-48b9-a2e9-b971b49afc84 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:53.492717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa7483-256b-48b9-a2e9-b971b49afc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:53.493000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa7483-256b-48b9-a2e9-b971b49afc84 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:53.493738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa7483-256b-48b9-a2e9-b971b49afc84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:49:53.494048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa7483-256b-48b9-a2e9-b971b49afc84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:53.495992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa7483-256b-48b9-a2e9-b971b49afc84 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:53.499217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f23e1-74df-499f-abc2-80236219eb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:53.499912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f23e1-74df-499f-abc2-80236219eb5d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:53.500199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f23e1-74df-499f-abc2-80236219eb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:53.500481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f23e1-74df-499f-abc2-80236219eb5d None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:53.501251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f23e1-74df-499f-abc2-80236219eb5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:49:53.501568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f23e1-74df-499f-abc2-80236219eb5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:53.503179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f23e1-74df-499f-abc2-80236219eb5d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:53.518345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 13/31] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:53.521872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 19/32] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:49:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:53.524355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa7483-256b-48b9-a2e9-b971b49afc84 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:53.527862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f23e1-74df-499f-abc2-80236219eb5d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:53.554850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:53.643121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa7483-256b-48b9-a2e9-b971b49afc84 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:53.663145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f23e1-74df-499f-abc2-80236219eb5d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:53.687442 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30e574-ab17-4b0d-a1dc-84f36f65f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:53.688122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30e574-ab17-4b0d-a1dc-84f36f65f155 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:53.688406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30e574-ab17-4b0d-a1dc-84f36f65f155 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:53.688688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30e574-ab17-4b0d-a1dc-84f36f65f155 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:53.689660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30e574-ab17-4b0d-a1dc-84f36f65f155 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:53.691658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30e574-ab17-4b0d-a1dc-84f36f65f155 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:53.694616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:53.706806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cb1d3-e05d-4ee8-9ade-d015c46e6f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:53.708019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb1d3-e05d-4ee8-9ade-d015c46e6f56 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:53.708516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb1d3-e05d-4ee8-9ade-d015c46e6f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:53.709020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cb1d3-e05d-4ee8-9ade-d015c46e6f56 None admin] PATH_INFO: `/v3/users/c2ba123046b04572883d482380082522` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:53.710567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cb1d3-e05d-4ee8-9ade-d015c46e6f56 None admin] RBAC: Authorizing `identity:get_user(user_id=c2ba123046b04572883d482380082522)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:53.712365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cb1d3-e05d-4ee8-9ade-d015c46e6f56 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:53.715021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 20/33] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:49:53 2022] GET /identity/v3/users/c2ba123046b04572883d482380082522 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:49:53.717706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda0f33-157b-4224-960a-8622a0ff0c15 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:53.718201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda0f33-157b-4224-960a-8622a0ff0c15 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:53.718840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda0f33-157b-4224-960a-8622a0ff0c15 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:53.720559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 21/34] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:49:53.721507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda0f33-157b-4224-960a-8622a0ff0c15 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:53.834937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda0f33-157b-4224-960a-8622a0ff0c15 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:53.850471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 14/35] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:49:53 2022] POST /identity/v3/projects => generated 301 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:49:53.850471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a813654-bae0-47fd-931f-1138fe835425 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:53.850471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a813654-bae0-47fd-931f-1138fe835425 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:53.850471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a813654-bae0-47fd-931f-1138fe835425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:53.868231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2a813654-bae0-47fd-931f-1138fe835425 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:53.869825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2a813654-bae0-47fd-931f-1138fe835425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:53.886798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459796ee-7bf1-4698-b7a8-526408bf3450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:53.888313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459796ee-7bf1-4698-b7a8-526408bf3450 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:53.888726 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459796ee-7bf1-4698-b7a8-526408bf3450 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:53.889114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459796ee-7bf1-4698-b7a8-526408bf3450 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:53.891080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459796ee-7bf1-4698-b7a8-526408bf3450 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:53.892657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459796ee-7bf1-4698-b7a8-526408bf3450 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:53.895690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 22/36] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:49:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:53.896959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e2593-95cf-4c25-b2c8-8607bd712bea None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:53.897308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e2593-95cf-4c25-b2c8-8607bd712bea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:53.897595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41e2593-95cf-4c25-b2c8-8607bd712bea None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:53.899126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 23/37] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:49:53.901014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e2593-95cf-4c25-b2c8-8607bd712bea None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:53.957642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:53.965089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 15/38] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:49:53.967861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97955a0b-2187-49cd-a3b0-e82b7f58e9d3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:53.968160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97955a0b-2187-49cd-a3b0-e82b7f58e9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:53.968450 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97955a0b-2187-49cd-a3b0-e82b7f58e9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:53.991031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:53.992611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-97955a0b-2187-49cd-a3b0-e82b7f58e9d3 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:53.994233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-97955a0b-2187-49cd-a3b0-e82b7f58e9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:54.035714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41e2593-95cf-4c25-b2c8-8607bd712bea None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:54.069957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 16/39] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:49:54.071780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e183b9d-3e23-4d46-88ab-cecf3e1d3a93 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:54.072184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e183b9d-3e23-4d46-88ab-cecf3e1d3a93 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:54.072470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e183b9d-3e23-4d46-88ab-cecf3e1d3a93 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:54.074608 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 17/40] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:49:54.075401 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e183b9d-3e23-4d46-88ab-cecf3e1d3a93 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:54.082000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a8270-3790-47a7-88e9-671a8b795cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:54.082966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a8270-3790-47a7-88e9-671a8b795cb2 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:54.083255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a8270-3790-47a7-88e9-671a8b795cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:54.083537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a8270-3790-47a7-88e9-671a8b795cb2 None admin] PATH_INFO: `/v3/domains/default/users/c2ba123046b04572883d482380082522/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:54.086272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a8270-3790-47a7-88e9-671a8b795cb2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c2ba123046b04572883d482380082522, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:54.088179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a8270-3790-47a7-88e9-671a8b795cb2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:54.105299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 24/41] 10.222.0.57 () {58 vars in 1361 bytes} [Thu May 26 20:49:53 2022] PUT /identity/v3/domains/default/users/c2ba123046b04572883d482380082522/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:49:54.106532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734a9bf-eabd-46ae-9017-188df64c7a7c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:54.106827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734a9bf-eabd-46ae-9017-188df64c7a7c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:54.107117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0734a9bf-eabd-46ae-9017-188df64c7a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:54.128716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0734a9bf-eabd-46ae-9017-188df64c7a7c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:54.131633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0734a9bf-eabd-46ae-9017-188df64c7a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:54.207818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e183b9d-3e23-4d46-88ab-cecf3e1d3a93 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:54.218606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 25/42] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:49:54.219982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734a9bf-eabd-46ae-9017-188df64c7a7c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:54.242143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:54.250701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd709a-1ca5-4d27-945e-7ac58fc7de8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:54.251391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd709a-1ca5-4d27-945e-7ac58fc7de8b None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:54.251686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd709a-1ca5-4d27-945e-7ac58fc7de8b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:54.251974 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd709a-1ca5-4d27-945e-7ac58fc7de8b None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:54.252738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd709a-1ca5-4d27-945e-7ac58fc7de8b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:54.254320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd709a-1ca5-4d27-945e-7ac58fc7de8b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:54.272005 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 18/43] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 26 20:49:54 2022] POST /identity/v3/roles => generated 225 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:49:54.273294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd709a-1ca5-4d27-945e-7ac58fc7de8b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:54.290514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:54.362592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0734a9bf-eabd-46ae-9017-188df64c7a7c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:54.413678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796f04a-d037-40ff-933b-d4ad830d6521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:54.414603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796f04a-d037-40ff-933b-d4ad830d6521 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:54.414895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796f04a-d037-40ff-933b-d4ad830d6521 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:54.415179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796f04a-d037-40ff-933b-d4ad830d6521 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:54.417214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796f04a-d037-40ff-933b-d4ad830d6521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:54.419122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796f04a-d037-40ff-933b-d4ad830d6521 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:54.421808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 26/44] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:49:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:54.427908 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796f04a-d037-40ff-933b-d4ad830d6521 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:54.433642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd709a-1ca5-4d27-945e-7ac58fc7de8b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:54.481407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca46bdb-4ab3-4e0d-9c98-d808ecdd2a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:54.482018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca46bdb-4ab3-4e0d-9c98-d808ecdd2a9d None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:54.482306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca46bdb-4ab3-4e0d-9c98-d808ecdd2a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:54.482817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca46bdb-4ab3-4e0d-9c98-d808ecdd2a9d None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:54.483571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca46bdb-4ab3-4e0d-9c98-d808ecdd2a9d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:54.484973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca46bdb-4ab3-4e0d-9c98-d808ecdd2a9d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:54.503275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 19/45] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 26 20:49:54 2022] POST /identity/v3/roles => generated 223 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:49:54.548756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796f04a-d037-40ff-933b-d4ad830d6521 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:54.581223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:54.601971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecf71f-9142-4eae-be5f-ff0ed5a54e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:54.602929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecf71f-9142-4eae-be5f-ff0ed5a54e4a None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:54.603390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecf71f-9142-4eae-be5f-ff0ed5a54e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:54.603816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecf71f-9142-4eae-be5f-ff0ed5a54e4a None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:54.604696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecf71f-9142-4eae-be5f-ff0ed5a54e4a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:54.606936 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecf71f-9142-4eae-be5f-ff0ed5a54e4a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:54.637462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 27/46] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:49:54 2022] POST /identity/v3/projects => generated 312 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:49:56.340993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92874d-b6a0-46ed-aa6c-7f1d80f189bc None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:56.341537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92874d-b6a0-46ed-aa6c-7f1d80f189bc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:56.341624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92874d-b6a0-46ed-aa6c-7f1d80f189bc None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:56.343030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 20/47] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:49:56.348283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e159dc-d8d5-4260-8902-4656dad42629 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:56.348665 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e159dc-d8d5-4260-8902-4656dad42629 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:56.348974 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e159dc-d8d5-4260-8902-4656dad42629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:56.365558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e8e159dc-d8d5-4260-8902-4656dad42629 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:56.366765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e8e159dc-d8d5-4260-8902-4656dad42629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:56.430592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 28/48] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:49:56.453837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92874d-b6a0-46ed-aa6c-7f1d80f189bc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:56.503857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:56.555856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92874d-b6a0-46ed-aa6c-7f1d80f189bc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:56.587897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b090513-e5b8-426a-a56c-3888d21f10d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:56.588690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b090513-e5b8-426a-a56c-3888d21f10d4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:56.588973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b090513-e5b8-426a-a56c-3888d21f10d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:56.589772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b090513-e5b8-426a-a56c-3888d21f10d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:56.599650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b090513-e5b8-426a-a56c-3888d21f10d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8b090513-e5b8-426a-a56c-3888d21f10d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:49:56.607783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:49:56.608755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:49:56.609536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 21/49] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:49:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:49:56.614651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e159dc-d8d5-4260-8902-4656dad42629 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:56.714831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e159dc-d8d5-4260-8902-4656dad42629 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:56.716326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:56.748828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf7233-1355-4d11-9ae1-44513f64dab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:56.749488 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7233-1355-4d11-9ae1-44513f64dab5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:56.749826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7233-1355-4d11-9ae1-44513f64dab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:56.750112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf7233-1355-4d11-9ae1-44513f64dab5 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:56.750898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf7233-1355-4d11-9ae1-44513f64dab5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:49:56.751207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf7233-1355-4d11-9ae1-44513f64dab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:56.752585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf7233-1355-4d11-9ae1-44513f64dab5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:56.760254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 29/50] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:49:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:56.766470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b090513-e5b8-426a-a56c-3888d21f10d4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:56.832550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dca88a-143a-4478-bbe9-da095e0ced8a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:56.832841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dca88a-143a-4478-bbe9-da095e0ced8a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:56.833122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dca88a-143a-4478-bbe9-da095e0ced8a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:56.834630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 30/51] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:49:56.840000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139d9aa-ce17-4a9b-bcc3-131c4ddf5647 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:56.840291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139d9aa-ce17-4a9b-bcc3-131c4ddf5647 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:56.840574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139d9aa-ce17-4a9b-bcc3-131c4ddf5647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:56.856887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4139d9aa-ce17-4a9b-bcc3-131c4ddf5647 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:56.858004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4139d9aa-ce17-4a9b-bcc3-131c4ddf5647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:56.868248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b090513-e5b8-426a-a56c-3888d21f10d4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:56.878748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:56.903925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bd88c-a7af-43fa-9065-443f5ed238f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:56.904797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd88c-a7af-43fa-9065-443f5ed238f2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:56.905085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd88c-a7af-43fa-9065-443f5ed238f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:56.905368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bd88c-a7af-43fa-9065-443f5ed238f2 None admin] PATH_INFO: `/v3/users/c2ba123046b04572883d482380082522` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:56.906807 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bd88c-a7af-43fa-9065-443f5ed238f2 None admin] RBAC: Authorizing `identity:get_user(user_id=c2ba123046b04572883d482380082522)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:56.908261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bd88c-a7af-43fa-9065-443f5ed238f2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:56.909883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 22/52] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:49:56 2022] GET /identity/v3/users/c2ba123046b04572883d482380082522 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:56.915276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bd88c-a7af-43fa-9065-443f5ed238f2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:56.924460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 31/53] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:49:56.946809 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139d9aa-ce17-4a9b-bcc3-131c4ddf5647 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:57.013443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bd88c-a7af-43fa-9065-443f5ed238f2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:57.036255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:57.043525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139d9aa-ce17-4a9b-bcc3-131c4ddf5647 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:57.046855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1176e4-1afd-496f-89d7-6468d082c597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:57.047658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1176e4-1afd-496f-89d7-6468d082c597 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:57.047955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1176e4-1afd-496f-89d7-6468d082c597 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:57.048244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1176e4-1afd-496f-89d7-6468d082c597 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:57.049437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1176e4-1afd-496f-89d7-6468d082c597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:57.050953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1176e4-1afd-496f-89d7-6468d082c597 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:57.052576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 23/54] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:49:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:49:57.058697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1176e4-1afd-496f-89d7-6468d082c597 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:57.078055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab17cb4-9899-4588-9663-8b29836fbc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:57.078888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab17cb4-9899-4588-9663-8b29836fbc18 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:57.079180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab17cb4-9899-4588-9663-8b29836fbc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:57.079462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab17cb4-9899-4588-9663-8b29836fbc18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:57.080644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab17cb4-9899-4588-9663-8b29836fbc18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:57.082117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab17cb4-9899-4588-9663-8b29836fbc18 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:57.083776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 32/55] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:49:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:49:57.089154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab17cb4-9899-4588-9663-8b29836fbc18 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:57.104198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:57.158034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1176e4-1afd-496f-89d7-6468d082c597 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:57.186336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab17cb4-9899-4588-9663-8b29836fbc18 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:57.190776 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ed5cd-af1e-4cac-a5cb-32c8649dec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:57.191434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ed5cd-af1e-4cac-a5cb-32c8649dec32 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:57.191726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ed5cd-af1e-4cac-a5cb-32c8649dec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:57.192008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ed5cd-af1e-4cac-a5cb-32c8649dec32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:57.192846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ed5cd-af1e-4cac-a5cb-32c8649dec32 None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.domain.id=default, user.id=c2ba123046b04572883d482380082522) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:49:57.193165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ed5cd-af1e-4cac-a5cb-32c8649dec32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:57.194639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ed5cd-af1e-4cac-a5cb-32c8649dec32 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:57.204044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 24/56] 10.222.0.57 () {56 vars in 1322 bytes} [Thu May 26 20:49:57 2022] GET /identity/v3/role_assignments?user.id=c2ba123046b04572883d482380082522&scope.domain.id=default&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:49:57.220312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682b96e-905d-4ff6-95fd-2fd967a0a6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:57.220995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682b96e-905d-4ff6-95fd-2fd967a0a6a1 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:57.221279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682b96e-905d-4ff6-95fd-2fd967a0a6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:57.221554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682b96e-905d-4ff6-95fd-2fd967a0a6a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:57.222299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682b96e-905d-4ff6-95fd-2fd967a0a6a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:57.223867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682b96e-905d-4ff6-95fd-2fd967a0a6a1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:57.251257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 33/57] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 26 20:49:57 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:49:59.434899 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85301a-d0bb-4d25-9f60-99c465cb087c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:59.435408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85301a-d0bb-4d25-9f60-99c465cb087c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:59.435512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85301a-d0bb-4d25-9f60-99c465cb087c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:59.436820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 25/58] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:49:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:49:59.442126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19c6d1-87a2-409c-8400-91e43b070e01 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:59.442515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19c6d1-87a2-409c-8400-91e43b070e01 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:59.442727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19c6d1-87a2-409c-8400-91e43b070e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:59.460242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5a19c6d1-87a2-409c-8400-91e43b070e01 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:49:59.461438 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5a19c6d1-87a2-409c-8400-91e43b070e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:49:59.470275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:59.526274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 34/59] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:49:59.551323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85301a-d0bb-4d25-9f60-99c465cb087c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:59.651781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85301a-d0bb-4d25-9f60-99c465cb087c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:59.661565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:59.685131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d9dbd-bf4b-43c2-867d-a52b76d70832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:59.685856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d9dbd-bf4b-43c2-867d-a52b76d70832 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:59.686142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d9dbd-bf4b-43c2-867d-a52b76d70832 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:59.686418 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d9dbd-bf4b-43c2-867d-a52b76d70832 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:59.687614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d9dbd-bf4b-43c2-867d-a52b76d70832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:59.689075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d9dbd-bf4b-43c2-867d-a52b76d70832 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:59.690704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 26/60] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:49:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:49:59.696134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19c6d1-87a2-409c-8400-91e43b070e01 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:49:59.791183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19c6d1-87a2-409c-8400-91e43b070e01 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:49:59.824260 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995d39c-a7a9-4a10-9953-59d5ddf67dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:49:59.824931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995d39c-a7a9-4a10-9953-59d5ddf67dec None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:49:59.825223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995d39c-a7a9-4a10-9953-59d5ddf67dec None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:49:59.825511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995d39c-a7a9-4a10-9953-59d5ddf67dec None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:49:59.826235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995d39c-a7a9-4a10-9953-59d5ddf67dec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:49:59.827756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1995d39c-a7a9-4a10-9953-59d5ddf67dec None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:49:59.848342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:49:59.852972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 35/61] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:49:59 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:03.156024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692b0ef-1bc3-403c-8a14-96e2ac824b90 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.156850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692b0ef-1bc3-403c-8a14-96e2ac824b90 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.157288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692b0ef-1bc3-403c-8a14-96e2ac824b90 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.158859 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 27/62] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:03.164344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fad6e2-0c86-4db4-b0bb-2269ae75b138 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.165108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fad6e2-0c86-4db4-b0bb-2269ae75b138 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.165540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fad6e2-0c86-4db4-b0bb-2269ae75b138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.188140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a8fad6e2-0c86-4db4-b0bb-2269ae75b138 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:03.189567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a8fad6e2-0c86-4db4-b0bb-2269ae75b138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:03.286328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 36/63] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:03.321819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692b0ef-1bc3-403c-8a14-96e2ac824b90 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:03.466889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692b0ef-1bc3-403c-8a14-96e2ac824b90 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:03.499449 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7189eb-6311-4215-9db3-b1c5bf0ba9b6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.500085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7189eb-6311-4215-9db3-b1c5bf0ba9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.500587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7189eb-6311-4215-9db3-b1c5bf0ba9b6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.502199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 37/64] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:03.507742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f7a1a-a399-439a-b2e8-91549acdc762 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.508335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f7a1a-a399-439a-b2e8-91549acdc762 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.508818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f7a1a-a399-439a-b2e8-91549acdc762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.525876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544163e1-5f21-4243-829d-99ab43cb0192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:03.527659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544163e1-5f21-4243-829d-99ab43cb0192 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.528167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544163e1-5f21-4243-829d-99ab43cb0192 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.528664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544163e1-5f21-4243-829d-99ab43cb0192 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.529129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dc8f7a1a-a399-439a-b2e8-91549acdc762 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:03.530700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dc8f7a1a-a399-439a-b2e8-91549acdc762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:03.559628 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544163e1-5f21-4243-829d-99ab43cb0192 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-544163e1-5f21-4243-829d-99ab43cb0192 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:03.575400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:03.576321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:03.577082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 28/65] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:03.584480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544163e1-5f21-4243-829d-99ab43cb0192 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:03.609806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 38/66] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:03.641569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6c36-cc84-4bf0-b835-f90a085d3ba6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.642207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6c36-cc84-4bf0-b835-f90a085d3ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.642716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6c36-cc84-4bf0-b835-f90a085d3ba6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.644316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 39/67] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:03.649765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a2d41-1e52-4476-b86c-41f74f5b1fb9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.650314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a2d41-1e52-4476-b86c-41f74f5b1fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.650853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a2d41-1e52-4476-b86c-41f74f5b1fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.666870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:03.668645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ee6a2d41-1e52-4476-b86c-41f74f5b1fb9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:03.670040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ee6a2d41-1e52-4476-b86c-41f74f5b1fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:03.728326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544163e1-5f21-4243-829d-99ab43cb0192 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:03.728873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:03.739093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 40/68] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:03.746773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a2d41-1e52-4476-b86c-41f74f5b1fb9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:03.768176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f87794-d52a-409c-81cb-200d0e17bb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:03.768845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f87794-d52a-409c-81cb-200d0e17bb2d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.769293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f87794-d52a-409c-81cb-200d0e17bb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.769741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f87794-d52a-409c-81cb-200d0e17bb2d None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.770833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f87794-d52a-409c-81cb-200d0e17bb2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:03.771300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f87794-d52a-409c-81cb-200d0e17bb2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:03.772838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f87794-d52a-409c-81cb-200d0e17bb2d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:03.786164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 29/69] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:03.787888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb93e-cdd0-4b0e-99b3-92b5f0910879 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.788322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb93e-cdd0-4b0e-99b3-92b5f0910879 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.788739 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bb93e-cdd0-4b0e-99b3-92b5f0910879 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.790101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 30/70] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:03.791004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bb93e-cdd0-4b0e-99b3-92b5f0910879 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:03.868677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a2d41-1e52-4476-b86c-41f74f5b1fb9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:03.901241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:03.905075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bb93e-cdd0-4b0e-99b3-92b5f0910879 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:03.927189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da1a33-0b8a-488d-b176-4adc915d90bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:03.928054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da1a33-0b8a-488d-b176-4adc915d90bd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.928341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da1a33-0b8a-488d-b176-4adc915d90bd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.928619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da1a33-0b8a-488d-b176-4adc915d90bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.940077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da1a33-0b8a-488d-b176-4adc915d90bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:03.942493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a38105-43ea-4165-8103-211b3e7ca608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:03.943303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38105-43ea-4165-8103-211b3e7ca608 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.943595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38105-43ea-4165-8103-211b3e7ca608 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.943878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38105-43ea-4165-8103-211b3e7ca608 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-22da1a33-0b8a-488d-b176-4adc915d90bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:03.949257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:03.950290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:03.951266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:03.955321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 41/71] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:50:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:03.956320 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da1a33-0b8a-488d-b176-4adc915d90bd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:03.957873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a38105-43ea-4165-8103-211b3e7ca608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b7a38105-43ea-4165-8103-211b3e7ca608 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:03.966090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:03.967219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:03.968148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 31/72] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:03.968927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d4628-de5f-4e89-9b56-8e9542633a76 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:03.968927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d4628-de5f-4e89-9b56-8e9542633a76 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:03.969060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d4628-de5f-4e89-9b56-8e9542633a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:03.989660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f50d4628-de5f-4e89-9b56-8e9542633a76 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:03.990835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f50d4628-de5f-4e89-9b56-8e9542633a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:04.020517 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:04.058987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 32/73] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:04.059828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d4628-de5f-4e89-9b56-8e9542633a76 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:04.071606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da1a33-0b8a-488d-b176-4adc915d90bd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:04.107061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f4247-4502-4953-9047-0c3138435d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:04.107930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f4247-4502-4953-9047-0c3138435d91 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.108213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f4247-4502-4953-9047-0c3138435d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.108487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f4247-4502-4953-9047-0c3138435d91 None admin] PATH_INFO: `/v3/users/9a041adb8e4f4e4abce713c554736b14` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.121929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f4247-4502-4953-9047-0c3138435d91 None admin] RBAC: Authorizing `identity:get_user(user_id=9a041adb8e4f4e4abce713c554736b14)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:04.123458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f4247-4502-4953-9047-0c3138435d91 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:04.125120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 42/74] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:03 2022] GET /identity/v3/users/9a041adb8e4f4e4abce713c554736b14 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:04.125899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f4247-4502-4953-9047-0c3138435d91 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:04.150111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:04.160575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d4628-de5f-4e89-9b56-8e9542633a76 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:04.194104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ff3ea-2c87-4d93-a47f-9d502531b4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:04.194814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ff3ea-2c87-4d93-a47f-9d502531b4fb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.195109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ff3ea-2c87-4d93-a47f-9d502531b4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.195394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ff3ea-2c87-4d93-a47f-9d502531b4fb None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.196144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ff3ea-2c87-4d93-a47f-9d502531b4fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:04.196461 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ff3ea-2c87-4d93-a47f-9d502531b4fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:04.197865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ff3ea-2c87-4d93-a47f-9d502531b4fb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:04.206520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 33/75] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:04.207731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3cc37-cc54-4483-90e7-948ac3888511 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.208027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3cc37-cc54-4483-90e7-948ac3888511 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.208314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3cc37-cc54-4483-90e7-948ac3888511 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.209550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 34/76] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:04.210252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3cc37-cc54-4483-90e7-948ac3888511 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:04.226574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f4247-4502-4953-9047-0c3138435d91 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:04.236168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:04.261135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d10cc1-6bed-40d3-92cc-1b87dbac029c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:04.261734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d10cc1-6bed-40d3-92cc-1b87dbac029c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.262022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d10cc1-6bed-40d3-92cc-1b87dbac029c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.262303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d10cc1-6bed-40d3-92cc-1b87dbac029c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.263351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d10cc1-6bed-40d3-92cc-1b87dbac029c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:04.263663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d10cc1-6bed-40d3-92cc-1b87dbac029c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:04.265041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d10cc1-6bed-40d3-92cc-1b87dbac029c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:04.272724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 43/77] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:04.273504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d10cc1-6bed-40d3-92cc-1b87dbac029c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:04.317565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d3cc37-cc54-4483-90e7-948ac3888511 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:04.349364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a06c1-0bb9-499a-ac17-f99c509a6880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:04.350156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a06c1-0bb9-499a-ac17-f99c509a6880 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.350535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a06c1-0bb9-499a-ac17-f99c509a6880 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.350753 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a06c1-0bb9-499a-ac17-f99c509a6880 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.361220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a06c1-0bb9-499a-ac17-f99c509a6880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:04.368234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d10cc1-6bed-40d3-92cc-1b87dbac029c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ca9a06c1-0bb9-499a-ac17-f99c509a6880 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:04.368985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:04.369817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:04.371218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 35/78] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:04.371838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a06c1-0bb9-499a-ac17-f99c509a6880 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:04.385888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:04.407248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c5005-ad7b-4133-bc6b-acac4b056d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:04.407898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c5005-ad7b-4133-bc6b-acac4b056d5c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.408184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c5005-ad7b-4133-bc6b-acac4b056d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.408461 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c5005-ad7b-4133-bc6b-acac4b056d5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.446775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:04.472391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a06c1-0bb9-499a-ac17-f99c509a6880 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:04.474073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c5005-ad7b-4133-bc6b-acac4b056d5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:04.475610 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c5005-ad7b-4133-bc6b-acac4b056d5c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:04.505482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2506625-6871-45ff-bb8e-cf06690f9b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:04.506348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2506625-6871-45ff-bb8e-cf06690f9b29 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.506671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2506625-6871-45ff-bb8e-cf06690f9b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.506950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2506625-6871-45ff-bb8e-cf06690f9b29 None admin] PATH_INFO: `/v3/users/c2ba123046b04572883d482380082522` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.508171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2506625-6871-45ff-bb8e-cf06690f9b29 None admin] RBAC: Authorizing `identity:get_user(user_id=c2ba123046b04572883d482380082522)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:04.509644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2506625-6871-45ff-bb8e-cf06690f9b29 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:04.511384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 36/79] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:04 2022] GET /identity/v3/users/c2ba123046b04572883d482380082522 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:04.512715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71510a27-b99a-4265-ab35-eda8ee835dc3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.513017 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71510a27-b99a-4265-ab35-eda8ee835dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.513306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71510a27-b99a-4265-ab35-eda8ee835dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.528989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-71510a27-b99a-4265-ab35-eda8ee835dc3 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:04.530167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-71510a27-b99a-4265-ab35-eda8ee835dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:04.573057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 44/80] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:04.574428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c5005-ad7b-4133-bc6b-acac4b056d5c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:04.599808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 37/81] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:04.600487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71510a27-b99a-4265-ab35-eda8ee835dc3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:04.611930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:04.652652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:04.686869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c5005-ad7b-4133-bc6b-acac4b056d5c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:04.700261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71510a27-b99a-4265-ab35-eda8ee835dc3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:04.720454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f628b-7af1-4509-8f80-e402d0b5d75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:04.721331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f628b-7af1-4509-8f80-e402d0b5d75a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.721616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f628b-7af1-4509-8f80-e402d0b5d75a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.721902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f628b-7af1-4509-8f80-e402d0b5d75a None admin] PATH_INFO: `/v3/users/9a041adb8e4f4e4abce713c554736b14` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.723100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f628b-7af1-4509-8f80-e402d0b5d75a None admin] RBAC: Authorizing `identity:get_user(user_id=9a041adb8e4f4e4abce713c554736b14)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:04.724580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f628b-7af1-4509-8f80-e402d0b5d75a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:04.726391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 45/82] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:04 2022] GET /identity/v3/users/9a041adb8e4f4e4abce713c554736b14 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:04.727016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f628b-7af1-4509-8f80-e402d0b5d75a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:04.738595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443aec1-185b-4778-b562-e1f2e646ef73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:04.739267 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443aec1-185b-4778-b562-e1f2e646ef73 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.739554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443aec1-185b-4778-b562-e1f2e646ef73 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.739835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443aec1-185b-4778-b562-e1f2e646ef73 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.740583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443aec1-185b-4778-b562-e1f2e646ef73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:04.740894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443aec1-185b-4778-b562-e1f2e646ef73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:04.742309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443aec1-185b-4778-b562-e1f2e646ef73 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:04.751104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 38/83] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:04.751887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443aec1-185b-4778-b562-e1f2e646ef73 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:04.825922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:04.829926 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f628b-7af1-4509-8f80-e402d0b5d75a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:04.855508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443aec1-185b-4778-b562-e1f2e646ef73 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:04.863274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a741ba6-8386-4444-b559-9213c9606ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:04.863923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a741ba6-8386-4444-b559-9213c9606ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.864213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a741ba6-8386-4444-b559-9213c9606ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.864494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a741ba6-8386-4444-b559-9213c9606ec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.872802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:04.889573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8177ce-f4ce-47ba-b255-3edf2d90f633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:04.890435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8177ce-f4ce-47ba-b255-3edf2d90f633 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:04.890760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8177ce-f4ce-47ba-b255-3edf2d90f633 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:04.891046 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8177ce-f4ce-47ba-b255-3edf2d90f633 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:04.892242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8177ce-f4ce-47ba-b255-3edf2d90f633 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:04.893701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8177ce-f4ce-47ba-b255-3edf2d90f633 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:04.895452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 39/84] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:04 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:04.896546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8177ce-f4ce-47ba-b255-3edf2d90f633 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:04.935950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a741ba6-8386-4444-b559-9213c9606ec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:04.937441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a741ba6-8386-4444-b559-9213c9606ec0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:04.999565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8177ce-f4ce-47ba-b255-3edf2d90f633 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:05.031956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194baf2-6299-43e6-8f91-a97a0dedf017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:05.032762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194baf2-6299-43e6-8f91-a97a0dedf017 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:05.033054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194baf2-6299-43e6-8f91-a97a0dedf017 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:05.033340 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194baf2-6299-43e6-8f91-a97a0dedf017 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:05.034557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194baf2-6299-43e6-8f91-a97a0dedf017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:05.036010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194baf2-6299-43e6-8f91-a97a0dedf017 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:05.037631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 40/85] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:05.038864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 46/86] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:05.039701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a741ba6-8386-4444-b559-9213c9606ec0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:05.043074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194baf2-6299-43e6-8f91-a97a0dedf017 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:05.053805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:05.080179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:05.138719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a741ba6-8386-4444-b559-9213c9606ec0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:05.140908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194baf2-6299-43e6-8f91-a97a0dedf017 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:05.171591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54246b-c448-4c06-ad91-a5509e2ebfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:05.172463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54246b-c448-4c06-ad91-a5509e2ebfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:05.172753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54246b-c448-4c06-ad91-a5509e2ebfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:05.173033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f54246b-c448-4c06-ad91-a5509e2ebfe2 None admin] PATH_INFO: `/v3/users/9a041adb8e4f4e4abce713c554736b14` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:05.173882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cd8f1-1a20-4e96-8c9b-f47796201941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:05.174313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54246b-c448-4c06-ad91-a5509e2ebfe2 None admin] RBAC: Authorizing `identity:get_user(user_id=9a041adb8e4f4e4abce713c554736b14)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:05.174566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cd8f1-1a20-4e96-8c9b-f47796201941 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:05.174867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cd8f1-1a20-4e96-8c9b-f47796201941 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:05.175153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cd8f1-1a20-4e96-8c9b-f47796201941 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:05.175701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f54246b-c448-4c06-ad91-a5509e2ebfe2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:05.177318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 47/87] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:05 2022] GET /identity/v3/users/9a041adb8e4f4e4abce713c554736b14 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:05.178076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54246b-c448-4c06-ad91-a5509e2ebfe2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:05.243240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cd8f1-1a20-4e96-8c9b-f47796201941 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:05.244713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cd8f1-1a20-4e96-8c9b-f47796201941 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:05.254840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:05.278070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f54246b-c448-4c06-ad91-a5509e2ebfe2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:05.286848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:05.315377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420f693-d05f-4dd8-bd00-dc292c0abe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:05.316033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420f693-d05f-4dd8-bd00-dc292c0abe8e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:05.316318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420f693-d05f-4dd8-bd00-dc292c0abe8e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:05.316595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420f693-d05f-4dd8-bd00-dc292c0abe8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:05.317427 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420f693-d05f-4dd8-bd00-dc292c0abe8e None admin] RBAC: Adding query filter params (role.id=d3bdf57150744faf9c5c25c317df48fc, scope.project.id=bb614e1d464d4df1959debd82ba5049c, user.id=9a041adb8e4f4e4abce713c554736b14) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:05.317742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420f693-d05f-4dd8-bd00-dc292c0abe8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:05.319248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420f693-d05f-4dd8-bd00-dc292c0abe8e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:05.324685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 48/88] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:05 2022] GET /identity/v3/role_assignments?user.id=9a041adb8e4f4e4abce713c554736b14&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:05.325444 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420f693-d05f-4dd8-bd00-dc292c0abe8e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:05.355240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 41/89] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:05.355979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cd8f1-1a20-4e96-8c9b-f47796201941 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:05.451883 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420f693-d05f-4dd8-bd00-dc292c0abe8e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:05.469206 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cd8f1-1a20-4e96-8c9b-f47796201941 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:05.479182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:05.489764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd0bf5-42cd-46fc-9bd9-4378ba383757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:05.490505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd0bf5-42cd-46fc-9bd9-4378ba383757 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:05.490802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd0bf5-42cd-46fc-9bd9-4378ba383757 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:05.491086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd0bf5-42cd-46fc-9bd9-4378ba383757 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:05.491819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd0bf5-42cd-46fc-9bd9-4378ba383757 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:05.493341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd0bf5-42cd-46fc-9bd9-4378ba383757 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:05.509011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93b62e-7d06-46c2-ab00-b011b4eb7ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:05.510219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93b62e-7d06-46c2-ab00-b011b4eb7ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:05.510219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93b62e-7d06-46c2-ab00-b011b4eb7ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:05.510505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93b62e-7d06-46c2-ab00-b011b4eb7ad9 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:05.511722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93b62e-7d06-46c2-ab00-b011b4eb7ad9 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:05.513566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a93b62e-7d06-46c2-ab00-b011b4eb7ad9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:05.515432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 42/90] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:05 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:05.515999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93b62e-7d06-46c2-ab00-b011b4eb7ad9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:05.520035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 49/91] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:50:05 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:05.520874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd0bf5-42cd-46fc-9bd9-4378ba383757 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:05.566868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:05.645535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93b62e-7d06-46c2-ab00-b011b4eb7ad9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:05.662046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd0bf5-42cd-46fc-9bd9-4378ba383757 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:05.686798 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b1489-fecb-4941-b49d-a43f074e4d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:05.687585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b1489-fecb-4941-b49d-a43f074e4d6e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:05.687979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b1489-fecb-4941-b49d-a43f074e4d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:05.688264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b1489-fecb-4941-b49d-a43f074e4d6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:05.700639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4b102-4ea6-49ed-8611-4eb949278d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:05.701531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4b102-4ea6-49ed-8611-4eb949278d87 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:05.701826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4b102-4ea6-49ed-8611-4eb949278d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:05.702112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4b102-4ea6-49ed-8611-4eb949278d87 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:05.703442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4b102-4ea6-49ed-8611-4eb949278d87 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:05.704965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4b102-4ea6-49ed-8611-4eb949278d87 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:05.706821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 50/92] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:05 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:05.712477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4b102-4ea6-49ed-8611-4eb949278d87 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:05.745619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:05.772827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b1489-fecb-4941-b49d-a43f074e4d6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:05.774606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b1489-fecb-4941-b49d-a43f074e4d6e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:05.833269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:05.841845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4b102-4ea6-49ed-8611-4eb949278d87 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:05.892414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76eec7-24fe-41d5-b4f0-946d13b3bdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:05.893356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76eec7-24fe-41d5-b4f0-946d13b3bdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:05.893795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76eec7-24fe-41d5-b4f0-946d13b3bdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:05.894221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76eec7-24fe-41d5-b4f0-946d13b3bdc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:05.895244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76eec7-24fe-41d5-b4f0-946d13b3bdc7 None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=bb614e1d464d4df1959debd82ba5049c, user.id=c2ba123046b04572883d482380082522) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:05.895753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76eec7-24fe-41d5-b4f0-946d13b3bdc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:05.897403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76eec7-24fe-41d5-b4f0-946d13b3bdc7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:05.903899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 51/93] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:05 2022] GET /identity/v3/role_assignments?user.id=c2ba123046b04572883d482380082522&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:05.904714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76eec7-24fe-41d5-b4f0-946d13b3bdc7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:05.909037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 43/94] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:05.915553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b1489-fecb-4941-b49d-a43f074e4d6e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:06.046424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:06.056207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76eec7-24fe-41d5-b4f0-946d13b3bdc7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:06.103572 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b1489-fecb-4941-b49d-a43f074e4d6e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:06.129826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce4af0-3e9d-48a3-ab77-d7ab0f19f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:06.130528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce4af0-3e9d-48a3-ab77-d7ab0f19f63c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:06.130821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce4af0-3e9d-48a3-ab77-d7ab0f19f63c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:06.131103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce4af0-3e9d-48a3-ab77-d7ab0f19f63c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:06.131957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce4af0-3e9d-48a3-ab77-d7ab0f19f63c None admin] RBAC: Adding query filter params (role.id=fa67cb2f211d433097f606ff029f863e, scope.project.id=bb614e1d464d4df1959debd82ba5049c, user.id=9a041adb8e4f4e4abce713c554736b14) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:06.132276 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce4af0-3e9d-48a3-ab77-d7ab0f19f63c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:06.134083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce4af0-3e9d-48a3-ab77-d7ab0f19f63c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:06.150716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 52/95] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:05 2022] GET /identity/v3/role_assignments?user.id=9a041adb8e4f4e4abce713c554736b14&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=fa67cb2f211d433097f606ff029f863e => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:06.163518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1c677-9749-4013-8b1e-b27ea36be5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:06.164428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1c677-9749-4013-8b1e-b27ea36be5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:06.164717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1c677-9749-4013-8b1e-b27ea36be5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:06.165002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1c677-9749-4013-8b1e-b27ea36be5e3 None admin] PATH_INFO: `/v3/projects/3ffcc0e7c1584efaa5fa4473241a9bf6` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:06.166394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1c677-9749-4013-8b1e-b27ea36be5e3 None admin] RBAC: Authorizing `identity:get_project(project_id=3ffcc0e7c1584efaa5fa4473241a9bf6)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:06.167942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a1c677-9749-4013-8b1e-b27ea36be5e3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:06.170160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 44/96] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:05 2022] GET /identity/v3/projects/3ffcc0e7c1584efaa5fa4473241a9bf6 => generated 312 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:06.175687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce4af0-3e9d-48a3-ab77-d7ab0f19f63c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:06.212555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:06.315612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce4af0-3e9d-48a3-ab77-d7ab0f19f63c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:06.374412 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116c3ee-73c5-4c5f-8a83-1913c2cc8946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:06.375111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116c3ee-73c5-4c5f-8a83-1913c2cc8946 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:06.375400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116c3ee-73c5-4c5f-8a83-1913c2cc8946 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:06.375683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116c3ee-73c5-4c5f-8a83-1913c2cc8946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:06.376536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116c3ee-73c5-4c5f-8a83-1913c2cc8946 None admin] RBAC: Adding query filter params (role.id=d3bdf57150744faf9c5c25c317df48fc, scope.project.id=3ffcc0e7c1584efaa5fa4473241a9bf6, user.id=9a041adb8e4f4e4abce713c554736b14) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:06.376856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116c3ee-73c5-4c5f-8a83-1913c2cc8946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:06.378399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116c3ee-73c5-4c5f-8a83-1913c2cc8946 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:06.388790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 53/97] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:06 2022] GET /identity/v3/role_assignments?user.id=9a041adb8e4f4e4abce713c554736b14&scope.project.id=3ffcc0e7c1584efaa5fa4473241a9bf6&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:08.203900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b1e4-bb19-4666-9427-f33b42861c4a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:08.204377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b1e4-bb19-4666-9427-f33b42861c4a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:08.204377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b1e4-bb19-4666-9427-f33b42861c4a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:08.205892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 45/98] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:08.211149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de776fe9-8f8d-46d3-ad0b-164c8ee3a0e4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:08.211557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de776fe9-8f8d-46d3-ad0b-164c8ee3a0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:08.211717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de776fe9-8f8d-46d3-ad0b-164c8ee3a0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:08.234385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-de776fe9-8f8d-46d3-ad0b-164c8ee3a0e4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:08.235624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-de776fe9-8f8d-46d3-ad0b-164c8ee3a0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:08.324531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 54/99] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:08.359032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086b1e4-bb19-4666-9427-f33b42861c4a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:08.492303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086b1e4-bb19-4666-9427-f33b42861c4a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:08.534718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd682e82-4c86-4195-9e3a-acfa44793fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:08.535528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd682e82-4c86-4195-9e3a-acfa44793fdd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:08.535815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd682e82-4c86-4195-9e3a-acfa44793fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:08.536094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd682e82-4c86-4195-9e3a-acfa44793fdd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:08.549179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd682e82-4c86-4195-9e3a-acfa44793fdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-dd682e82-4c86-4195-9e3a-acfa44793fdd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:08.558823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:08.559755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:08.560505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 46/100] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:08.567706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de776fe9-8f8d-46d3-ad0b-164c8ee3a0e4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:08.587167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:08.642484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6398271-387c-4719-9fd2-918959975a32 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:08.642935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6398271-387c-4719-9fd2-918959975a32 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:08.643314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6398271-387c-4719-9fd2-918959975a32 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:08.650537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 47/101] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:08.660063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834294ce-4b46-45cc-be90-22dc0b3f161b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:08.660475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834294ce-4b46-45cc-be90-22dc0b3f161b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:08.660879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834294ce-4b46-45cc-be90-22dc0b3f161b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:08.671891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:08.688260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-834294ce-4b46-45cc-be90-22dc0b3f161b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:08.689612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-834294ce-4b46-45cc-be90-22dc0b3f161b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:08.691297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de776fe9-8f8d-46d3-ad0b-164c8ee3a0e4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:08.728051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7050913-02a7-4393-aa43-008d0a2dcbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:08.729124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7050913-02a7-4393-aa43-008d0a2dcbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:08.729628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7050913-02a7-4393-aa43-008d0a2dcbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:08.730108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7050913-02a7-4393-aa43-008d0a2dcbe1 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:08.731069 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7050913-02a7-4393-aa43-008d0a2dcbe1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:08.731569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7050913-02a7-4393-aa43-008d0a2dcbe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:08.733262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7050913-02a7-4393-aa43-008d0a2dcbe1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:08.742213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 55/102] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:08.753237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7050913-02a7-4393-aa43-008d0a2dcbe1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:08.795700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 48/103] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:08.818544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834294ce-4b46-45cc-be90-22dc0b3f161b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:08.919849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:08.921071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834294ce-4b46-45cc-be90-22dc0b3f161b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:08.956408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7050913-02a7-4393-aa43-008d0a2dcbe1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:08.963487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef96f66-f249-4214-ba74-93a1119d2845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:08.964853 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef96f66-f249-4214-ba74-93a1119d2845 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:08.965138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef96f66-f249-4214-ba74-93a1119d2845 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:08.965416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef96f66-f249-4214-ba74-93a1119d2845 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:08.967723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:08.978781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef96f66-f249-4214-ba74-93a1119d2845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0ef96f66-f249-4214-ba74-93a1119d2845 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:08.987035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:08.987903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:08.988658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:08.991075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 49/104] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:08.992581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc66372-32ee-4829-a226-89d1c8775d03 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:08.993030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc66372-32ee-4829-a226-89d1c8775d03 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:08.993465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc66372-32ee-4829-a226-89d1c8775d03 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:08.995027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 50/105] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:09.000614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f9715-0c13-4cca-8d0f-3b2a0ad72814 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.001288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f9715-0c13-4cca-8d0f-3b2a0ad72814 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.003002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f9715-0c13-4cca-8d0f-3b2a0ad72814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.015813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3633f8-6082-405e-ac1b-3929e088f335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:09.018173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3633f8-6082-405e-ac1b-3929e088f335 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.018745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3633f8-6082-405e-ac1b-3929e088f335 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.020442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3633f8-6082-405e-ac1b-3929e088f335 None admin] PATH_INFO: `/v3/users/9a041adb8e4f4e4abce713c554736b14` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.022297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3633f8-6082-405e-ac1b-3929e088f335 None admin] RBAC: Authorizing `identity:get_user(user_id=9a041adb8e4f4e4abce713c554736b14)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:09.027195 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3633f8-6082-405e-ac1b-3929e088f335 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:09.037998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 56/106] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:08 2022] GET /identity/v3/users/9a041adb8e4f4e4abce713c554736b14 => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:09.039116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3633f8-6082-405e-ac1b-3929e088f335 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:09.043112 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-446f9715-0c13-4cca-8d0f-3b2a0ad72814 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:09.045257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-446f9715-0c13-4cca-8d0f-3b2a0ad72814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:09.171226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 51/107] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:09.172208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f9715-0c13-4cca-8d0f-3b2a0ad72814 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:09.222020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3633f8-6082-405e-ac1b-3929e088f335 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:09.269103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8035a-a365-4367-9717-b864aa9401fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:09.269723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8035a-a365-4367-9717-b864aa9401fd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.270013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8035a-a365-4367-9717-b864aa9401fd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.270313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8035a-a365-4367-9717-b864aa9401fd None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.271100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8035a-a365-4367-9717-b864aa9401fd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:09.271414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8035a-a365-4367-9717-b864aa9401fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:09.272875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8035a-a365-4367-9717-b864aa9401fd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:09.294223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 57/108] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:09.296403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8035a-a365-4367-9717-b864aa9401fd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:09.319957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:09.357443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f9715-0c13-4cca-8d0f-3b2a0ad72814 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:09.401883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:09.421394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14d6d1-ac83-4631-98ce-2014556f2410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:09.422270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14d6d1-ac83-4631-98ce-2014556f2410 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.422580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14d6d1-ac83-4631-98ce-2014556f2410 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.422858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14d6d1-ac83-4631-98ce-2014556f2410 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.424179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14d6d1-ac83-4631-98ce-2014556f2410 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:09.425653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14d6d1-ac83-4631-98ce-2014556f2410 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:09.429802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 52/109] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:09 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:09.430581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14d6d1-ac83-4631-98ce-2014556f2410 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:09.465456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8035a-a365-4367-9717-b864aa9401fd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:09.502218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f1c3a-caa4-47d5-85a5-6c4982dc1096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:09.503078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f1c3a-caa4-47d5-85a5-6c4982dc1096 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.503368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f1c3a-caa4-47d5-85a5-6c4982dc1096 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.503658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f1c3a-caa4-47d5-85a5-6c4982dc1096 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.505027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f1c3a-caa4-47d5-85a5-6c4982dc1096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:09.506717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f1c3a-caa4-47d5-85a5-6c4982dc1096 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:09.508572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 58/110] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:09.514185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f1c3a-caa4-47d5-85a5-6c4982dc1096 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:09.573595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14d6d1-ac83-4631-98ce-2014556f2410 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:09.608877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914fc867-1e04-486e-9b2d-6b7dea94dfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:09.609795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc867-1e04-486e-9b2d-6b7dea94dfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.610030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc867-1e04-486e-9b2d-6b7dea94dfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.610310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914fc867-1e04-486e-9b2d-6b7dea94dfe0 None admin] PATH_INFO: `/v3/users/81b63e4c88d843ffb38c72010e564bba` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.618298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:09.626722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fc867-1e04-486e-9b2d-6b7dea94dfe0 None admin] RBAC: Authorizing `identity:get_user(user_id=81b63e4c88d843ffb38c72010e564bba)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:09.628193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914fc867-1e04-486e-9b2d-6b7dea94dfe0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:09.629963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 53/111] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:09 2022] GET /identity/v3/users/81b63e4c88d843ffb38c72010e564bba => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:09.631248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887a0a7-b919-4e83-a128-396aaddecf83 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.631541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887a0a7-b919-4e83-a128-396aaddecf83 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.631884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887a0a7-b919-4e83-a128-396aaddecf83 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.633184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 54/112] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:09.637623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887a0a7-b919-4e83-a128-396aaddecf83 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:09.654810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f1c3a-caa4-47d5-85a5-6c4982dc1096 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:09.697248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41007792-8db7-4337-9f97-fe94d04b417d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:09.698701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41007792-8db7-4337-9f97-fe94d04b417d None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.699035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41007792-8db7-4337-9f97-fe94d04b417d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.699320 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41007792-8db7-4337-9f97-fe94d04b417d None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c/users/9a041adb8e4f4e4abce713c554736b14/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.700117 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:09.702730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41007792-8db7-4337-9f97-fe94d04b417d None admin] RBAC: Authorizing `identity:create_grant(project_id=bb614e1d464d4df1959debd82ba5049c, user_id=9a041adb8e4f4e4abce713c554736b14, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:09.704541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41007792-8db7-4337-9f97-fe94d04b417d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:09.718551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 59/113] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:50:09 2022] PUT /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c/users/9a041adb8e4f4e4abce713c554736b14/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:50:09.719581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41007792-8db7-4337-9f97-fe94d04b417d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:09.770797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887a0a7-b919-4e83-a128-396aaddecf83 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:09.811684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ae393-5cd1-4f6c-ac42-5232c22be114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:09.812949 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ae393-5cd1-4f6c-ac42-5232c22be114 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.813438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ae393-5cd1-4f6c-ac42-5232c22be114 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.813913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ae393-5cd1-4f6c-ac42-5232c22be114 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.814868 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ae393-5cd1-4f6c-ac42-5232c22be114 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:09.817468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ae393-5cd1-4f6c-ac42-5232c22be114 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:09.849225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 55/114] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:50:09 2022] POST /identity/v3/projects => generated 302 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:09.851517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6b522-5b9b-4603-9df5-2aaae64073b7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.852242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6b522-5b9b-4603-9df5-2aaae64073b7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.852728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6b522-5b9b-4603-9df5-2aaae64073b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.858247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41007792-8db7-4337-9f97-fe94d04b417d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:09.876244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8ee6b522-5b9b-4603-9df5-2aaae64073b7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:09.878027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8ee6b522-5b9b-4603-9df5-2aaae64073b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:09.927131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d18d33-3998-4ab4-a532-0f325c4b07e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:09.927811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d18d33-3998-4ab4-a532-0f325c4b07e6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.928103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d18d33-3998-4ab4-a532-0f325c4b07e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.928394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d18d33-3998-4ab4-a532-0f325c4b07e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.933359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:09.982396 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 56/115] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:09.984035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cce2e-02e5-43ac-841e-00de3820fec2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.984334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cce2e-02e5-43ac-841e-00de3820fec2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.984617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cce2e-02e5-43ac-841e-00de3820fec2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.986644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 57/116] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:09.988323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb99bef-d0b4-493b-a628-a3f52249b096 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.988619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb99bef-d0b4-493b-a628-a3f52249b096 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.988904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb99bef-d0b4-493b-a628-a3f52249b096 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:09.990992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 58/117] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:09.996009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6d6a7-f7a9-4244-91a9-ffa2e181b2f7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:09.996282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6d6a7-f7a9-4244-91a9-ffa2e181b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:09.996580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6d6a7-f7a9-4244-91a9-ffa2e181b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:10.014475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e9a6d6a7-f7a9-4244-91a9-ffa2e181b2f7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:10.019209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e9a6d6a7-f7a9-4244-91a9-ffa2e181b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:10.029452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d18d33-3998-4ab4-a532-0f325c4b07e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:10.031465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d18d33-3998-4ab4-a532-0f325c4b07e6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:10.050500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:10.102186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 59/118] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:10.104029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3462c0-c5a8-4c65-a6db-b14708ad5986 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:10.104380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3462c0-c5a8-4c65-a6db-b14708ad5986 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:10.104800 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3462c0-c5a8-4c65-a6db-b14708ad5986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:10.127996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8f3462c0-c5a8-4c65-a6db-b14708ad5986 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:10.129272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8f3462c0-c5a8-4c65-a6db-b14708ad5986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:10.164421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 60/119] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:10.165316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d18d33-3998-4ab4-a532-0f325c4b07e6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:10.207068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:10.213307 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 60/120] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:10.214553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3462c0-c5a8-4c65-a6db-b14708ad5986 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:10.290565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d18d33-3998-4ab4-a532-0f325c4b07e6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:10.321363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:10.330181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e4255-1dec-421a-a3af-f2034f168ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:10.331884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3462c0-c5a8-4c65-a6db-b14708ad5986 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:10.332017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e4255-1dec-421a-a3af-f2034f168ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:10.332435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e4255-1dec-421a-a3af-f2034f168ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:10.332861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e4255-1dec-421a-a3af-f2034f168ac4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:10.346116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e4255-1dec-421a-a3af-f2034f168ac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-510e4255-1dec-421a-a3af-f2034f168ac4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:10.355970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:10.356887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:10.357684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:10.360246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 61/121] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:10.361747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e4255-1dec-421a-a3af-f2034f168ac4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:10.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05318cdd-1a63-4f0b-a9d5-0a3ddf98bb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:10.389793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05318cdd-1a63-4f0b-a9d5-0a3ddf98bb47 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:10.390286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05318cdd-1a63-4f0b-a9d5-0a3ddf98bb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:10.390759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05318cdd-1a63-4f0b-a9d5-0a3ddf98bb47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:10.404830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05318cdd-1a63-4f0b-a9d5-0a3ddf98bb47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-05318cdd-1a63-4f0b-a9d5-0a3ddf98bb47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:10.414942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:10.415770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:10.416537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:10.424757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 61/122] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:50:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:10.424757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05318cdd-1a63-4f0b-a9d5-0a3ddf98bb47 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:10.475508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:10.482944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e4255-1dec-421a-a3af-f2034f168ac4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:10.520011 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b87c0-e303-4a42-8dcc-3a76be083b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:10.521284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b87c0-e303-4a42-8dcc-3a76be083b97 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:10.521775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b87c0-e303-4a42-8dcc-3a76be083b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:10.522360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b87c0-e303-4a42-8dcc-3a76be083b97 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:10.523820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b87c0-e303-4a42-8dcc-3a76be083b97 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:10.525816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b87c0-e303-4a42-8dcc-3a76be083b97 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:10.527816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 62/123] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:10 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:10.528898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b87c0-e303-4a42-8dcc-3a76be083b97 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:10.540189 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05318cdd-1a63-4f0b-a9d5-0a3ddf98bb47 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:10.578037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0370a-aa2e-416b-9fd0-38a8a0393711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:10.578895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0370a-aa2e-416b-9fd0-38a8a0393711 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:10.579486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0370a-aa2e-416b-9fd0-38a8a0393711 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:10.579598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0370a-aa2e-416b-9fd0-38a8a0393711 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:10.591300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f0370a-aa2e-416b-9fd0-38a8a0393711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-69f0370a-aa2e-416b-9fd0-38a8a0393711 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:10.599864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:10.600733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:10.601526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 62/124] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:10.602171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0370a-aa2e-416b-9fd0-38a8a0393711 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:10.618610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:10.651716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b87c0-e303-4a42-8dcc-3a76be083b97 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:10.689780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29d3ca-d501-4fd5-86eb-9e65c8bd985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:10.690915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29d3ca-d501-4fd5-86eb-9e65c8bd985c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:10.691465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29d3ca-d501-4fd5-86eb-9e65c8bd985c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:10.691948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29d3ca-d501-4fd5-86eb-9e65c8bd985c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:10.692901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29d3ca-d501-4fd5-86eb-9e65c8bd985c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:10.693507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29d3ca-d501-4fd5-86eb-9e65c8bd985c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:10.695210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29d3ca-d501-4fd5-86eb-9e65c8bd985c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:10.704229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 63/125] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:10.705498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29d3ca-d501-4fd5-86eb-9e65c8bd985c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:10.710252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0370a-aa2e-416b-9fd0-38a8a0393711 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:10.736213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:10.749097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528cadd-7bf4-4de4-aa87-d8db4f191e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:10.749787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528cadd-7bf4-4de4-aa87-d8db4f191e50 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:10.750078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528cadd-7bf4-4de4-aa87-d8db4f191e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:10.750363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528cadd-7bf4-4de4-aa87-d8db4f191e50 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:10.751472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528cadd-7bf4-4de4-aa87-d8db4f191e50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:10.751587 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528cadd-7bf4-4de4-aa87-d8db4f191e50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:10.753045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528cadd-7bf4-4de4-aa87-d8db4f191e50 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:10.766872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 63/126] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:10.766872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528cadd-7bf4-4de4-aa87-d8db4f191e50 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:10.822630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29d3ca-d501-4fd5-86eb-9e65c8bd985c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:10.843862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:10.862232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba67a8b-0b3a-434d-8548-5c7232cee1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:10.863352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba67a8b-0b3a-434d-8548-5c7232cee1aa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:10.863854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba67a8b-0b3a-434d-8548-5c7232cee1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:10.864353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba67a8b-0b3a-434d-8548-5c7232cee1aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:10.865578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba67a8b-0b3a-434d-8548-5c7232cee1aa None admin] RBAC: Adding query filter params (role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc, scope.project.id=bb614e1d464d4df1959debd82ba5049c, user.id=81b63e4c88d843ffb38c72010e564bba) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:10.866098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba67a8b-0b3a-434d-8548-5c7232cee1aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:10.867983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba67a8b-0b3a-434d-8548-5c7232cee1aa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:10.871468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528cadd-7bf4-4de4-aa87-d8db4f191e50 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:10.874464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 64/127] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:10 2022] GET /identity/v3/role_assignments?user.id=81b63e4c88d843ffb38c72010e564bba&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:10.875364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba67a8b-0b3a-434d-8548-5c7232cee1aa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:10.913436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c052b-5731-4dc5-8509-e1470da7145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:10.914221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c052b-5731-4dc5-8509-e1470da7145c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:10.914472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c052b-5731-4dc5-8509-e1470da7145c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:10.914779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c052b-5731-4dc5-8509-e1470da7145c None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:10.915520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c052b-5731-4dc5-8509-e1470da7145c None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:10.915839 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c052b-5731-4dc5-8509-e1470da7145c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:10.918132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c052b-5731-4dc5-8509-e1470da7145c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:10.927259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 64/128] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:10.928418 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c052b-5731-4dc5-8509-e1470da7145c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:10.987518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:10.999074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba67a8b-0b3a-434d-8548-5c7232cee1aa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:11.052573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e991b86-587d-46bd-bdbd-1e62a4a8d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:11.053885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e991b86-587d-46bd-bdbd-1e62a4a8d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:11.054376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e991b86-587d-46bd-bdbd-1e62a4a8d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:11.054888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e991b86-587d-46bd-bdbd-1e62a4a8d6f2 None admin] PATH_INFO: `/v3/users/9a041adb8e4f4e4abce713c554736b14` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:11.056395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e991b86-587d-46bd-bdbd-1e62a4a8d6f2 None admin] RBAC: Authorizing `identity:get_user(user_id=9a041adb8e4f4e4abce713c554736b14)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:11.058192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e991b86-587d-46bd-bdbd-1e62a4a8d6f2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:11.061001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 65/129] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:10 2022] GET /identity/v3/users/9a041adb8e4f4e4abce713c554736b14 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:11.062345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e991b86-587d-46bd-bdbd-1e62a4a8d6f2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:11.131985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c052b-5731-4dc5-8509-e1470da7145c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:11.175599 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b7962-4f6d-4614-a93b-935de9811b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:11.176481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b7962-4f6d-4614-a93b-935de9811b36 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:11.176768 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b7962-4f6d-4614-a93b-935de9811b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:11.177054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b7962-4f6d-4614-a93b-935de9811b36 None admin] PATH_INFO: `/v3/users/c2ba123046b04572883d482380082522` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:11.179143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b7962-4f6d-4614-a93b-935de9811b36 None admin] RBAC: Authorizing `identity:get_user(user_id=c2ba123046b04572883d482380082522)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:11.180673 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b7962-4f6d-4614-a93b-935de9811b36 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:11.182345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 65/130] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:10 2022] GET /identity/v3/users/c2ba123046b04572883d482380082522 => generated 247 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:11.183165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b7962-4f6d-4614-a93b-935de9811b36 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:11.184642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e991b86-587d-46bd-bdbd-1e62a4a8d6f2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:11.194808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:11.224894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfff20-0183-44e3-9403-c7d21adeaa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:11.226132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfff20-0183-44e3-9403-c7d21adeaa4d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:11.226783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfff20-0183-44e3-9403-c7d21adeaa4d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:11.227261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfff20-0183-44e3-9403-c7d21adeaa4d None admin] PATH_INFO: `/v3/users/9a041adb8e4f4e4abce713c554736b14` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:11.228672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfff20-0183-44e3-9403-c7d21adeaa4d None admin] RBAC: Authorizing `identity:get_user(user_id=9a041adb8e4f4e4abce713c554736b14)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:11.231486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfff20-0183-44e3-9403-c7d21adeaa4d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:11.233811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 66/131] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:11 2022] GET /identity/v3/users/9a041adb8e4f4e4abce713c554736b14 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:11.234972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfff20-0183-44e3-9403-c7d21adeaa4d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:11.270876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:11.349621 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfff20-0183-44e3-9403-c7d21adeaa4d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:11.385821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cdaf9-1114-49b7-9f15-adc8a164a5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:11.386642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cdaf9-1114-49b7-9f15-adc8a164a5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:11.386933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cdaf9-1114-49b7-9f15-adc8a164a5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:11.387217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cdaf9-1114-49b7-9f15-adc8a164a5f0 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:11.388663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cdaf9-1114-49b7-9f15-adc8a164a5f0 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:11.390286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cdaf9-1114-49b7-9f15-adc8a164a5f0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:11.392646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 67/132] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:11 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:11.393507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cdaf9-1114-49b7-9f15-adc8a164a5f0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:11.499783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cdaf9-1114-49b7-9f15-adc8a164a5f0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:11.502877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:11.524420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b7962-4f6d-4614-a93b-935de9811b36 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:11.538575 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c94d69-62d8-48ad-92b5-5bdf9b9bef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:11.541107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c94d69-62d8-48ad-92b5-5bdf9b9bef7d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:11.541107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c94d69-62d8-48ad-92b5-5bdf9b9bef7d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:11.541107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c94d69-62d8-48ad-92b5-5bdf9b9bef7d None admin] PATH_INFO: `/v3/projects/3ffcc0e7c1584efaa5fa4473241a9bf6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:11.543002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c94d69-62d8-48ad-92b5-5bdf9b9bef7d None admin] RBAC: Authorizing `identity:get_project(project_id=3ffcc0e7c1584efaa5fa4473241a9bf6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:11.550486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c94d69-62d8-48ad-92b5-5bdf9b9bef7d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:11.550486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 68/133] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:11 2022] GET /identity/v3/projects/3ffcc0e7c1584efaa5fa4473241a9bf6 => generated 312 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:11.550486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c94d69-62d8-48ad-92b5-5bdf9b9bef7d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:11.641963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813f85-e6d6-4295-ba70-7ed2fee47b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:11.643146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813f85-e6d6-4295-ba70-7ed2fee47b8a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:11.643604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813f85-e6d6-4295-ba70-7ed2fee47b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:11.644041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813f85-e6d6-4295-ba70-7ed2fee47b8a None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:11.647180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8813f85-e6d6-4295-ba70-7ed2fee47b8a None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:11.648828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8813f85-e6d6-4295-ba70-7ed2fee47b8a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:11.657345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c94d69-62d8-48ad-92b5-5bdf9b9bef7d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:11.657848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 66/134] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:11 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:11.658847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813f85-e6d6-4295-ba70-7ed2fee47b8a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:11.696947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099e08c-e21e-40f6-98f6-c4cd7986f5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:11.698801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099e08c-e21e-40f6-98f6-c4cd7986f5cc None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:11.699316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099e08c-e21e-40f6-98f6-c4cd7986f5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:11.699719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099e08c-e21e-40f6-98f6-c4cd7986f5cc None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c/users/c2ba123046b04572883d482380082522/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:11.707477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099e08c-e21e-40f6-98f6-c4cd7986f5cc None admin] RBAC: Authorizing `identity:create_grant(project_id=bb614e1d464d4df1959debd82ba5049c, user_id=c2ba123046b04572883d482380082522, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:11.708961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099e08c-e21e-40f6-98f6-c4cd7986f5cc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:11.736504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 69/135] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:50:11 2022] PUT /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c/users/c2ba123046b04572883d482380082522/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:50:11.737692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099e08c-e21e-40f6-98f6-c4cd7986f5cc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:11.794826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:11.839481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:11.892912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813f85-e6d6-4295-ba70-7ed2fee47b8a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:11.934628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099e08c-e21e-40f6-98f6-c4cd7986f5cc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:11.973856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26272235-642b-4662-aee7-a51b86a877be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:11.975192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272235-642b-4662-aee7-a51b86a877be None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:11.975480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272235-642b-4662-aee7-a51b86a877be None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:11.975774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26272235-642b-4662-aee7-a51b86a877be None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c/users/9a041adb8e4f4e4abce713c554736b14/roles/fa67cb2f211d433097f606ff029f863e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:11.978473 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26272235-642b-4662-aee7-a51b86a877be None admin] RBAC: Authorizing `identity:create_grant(project_id=bb614e1d464d4df1959debd82ba5049c, user_id=9a041adb8e4f4e4abce713c554736b14, role_id=fa67cb2f211d433097f606ff029f863e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:11.980149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26272235-642b-4662-aee7-a51b86a877be None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:11.990261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72f201-c4c5-4da2-b089-729327bfcfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:11.991683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72f201-c4c5-4da2-b089-729327bfcfcd None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:11.992198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72f201-c4c5-4da2-b089-729327bfcfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:11.992707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72f201-c4c5-4da2-b089-729327bfcfcd None admin] PATH_INFO: `/v3/projects/3ffcc0e7c1584efaa5fa4473241a9bf6/users/9a041adb8e4f4e4abce713c554736b14/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:11.995976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 70/136] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:50:11 2022] PUT /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c/users/9a041adb8e4f4e4abce713c554736b14/roles/fa67cb2f211d433097f606ff029f863e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:50:12.003607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72f201-c4c5-4da2-b089-729327bfcfcd None admin] RBAC: Authorizing `identity:create_grant(project_id=3ffcc0e7c1584efaa5fa4473241a9bf6, user_id=9a041adb8e4f4e4abce713c554736b14, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:12.005503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72f201-c4c5-4da2-b089-729327bfcfcd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:12.044216 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 67/137] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:50:11 2022] PUT /identity/v3/projects/3ffcc0e7c1584efaa5fa4473241a9bf6/users/9a041adb8e4f4e4abce713c554736b14/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:50:12.483876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b3244-ce91-4409-b8ef-7559a13a7f58 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:12.484395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b3244-ce91-4409-b8ef-7559a13a7f58 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:12.484469 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b3244-ce91-4409-b8ef-7559a13a7f58 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:12.485921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 71/138] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:12.491556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b2ed7-247f-4d66-a565-0234313f0581 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:12.492110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b2ed7-247f-4d66-a565-0234313f0581 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:12.492607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b2ed7-247f-4d66-a565-0234313f0581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:12.511807 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8a4b2ed7-247f-4d66-a565-0234313f0581 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:12.514279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbe7f9-10d5-4f85-9134-f119f0d15287 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:12.518670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbe7f9-10d5-4f85-9134-f119f0d15287 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:12.518977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbe7f9-10d5-4f85-9134-f119f0d15287 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:12.519591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8a4b2ed7-247f-4d66-a565-0234313f0581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:12.520359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 72/139] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:12.527542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b592582-6897-41c7-a543-46c44a93a245 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:12.527840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b592582-6897-41c7-a543-46c44a93a245 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:12.528125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b592582-6897-41c7-a543-46c44a93a245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:12.559033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5b592582-6897-41c7-a543-46c44a93a245 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:12.560248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5b592582-6897-41c7-a543-46c44a93a245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:12.644456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 68/140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:12.690694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 73/141] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:12.705522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b2ed7-247f-4d66-a565-0234313f0581 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:12.727632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b592582-6897-41c7-a543-46c44a93a245 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:12.749203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:12.842596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:12.857795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b592582-6897-41c7-a543-46c44a93a245 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:12.898472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d475f-0521-435d-bfe0-6e913e717b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:12.898472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d475f-0521-435d-bfe0-6e913e717b37 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:12.898472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d475f-0521-435d-bfe0-6e913e717b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:12.898472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d475f-0521-435d-bfe0-6e913e717b37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:12.898883 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d475f-0521-435d-bfe0-6e913e717b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:12.899140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d475f-0521-435d-bfe0-6e913e717b37 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:12.900685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 74/142] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:12.905835 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d475f-0521-435d-bfe0-6e913e717b37 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:12.912494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b2ed7-247f-4d66-a565-0234313f0581 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:12.951443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6635fe-a3ed-47d1-a8fe-a1a5544dc8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:12.952625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6635fe-a3ed-47d1-a8fe-a1a5544dc8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:12.952739 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6635fe-a3ed-47d1-a8fe-a1a5544dc8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:12.953136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6635fe-a3ed-47d1-a8fe-a1a5544dc8b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:12.972903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6635fe-a3ed-47d1-a8fe-a1a5544dc8b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2b6635fe-a3ed-47d1-a8fe-a1a5544dc8b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:12.981106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:12.981952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:12.982855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 69/143] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:12.991366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6635fe-a3ed-47d1-a8fe-a1a5544dc8b5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:13.041363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:13.069183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d475f-0521-435d-bfe0-6e913e717b37 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:13.122422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21879ec0-3ebb-4bc6-aeba-47dfd77655d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:13.123497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21879ec0-3ebb-4bc6-aeba-47dfd77655d3 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:13.123977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21879ec0-3ebb-4bc6-aeba-47dfd77655d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:13.124443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21879ec0-3ebb-4bc6-aeba-47dfd77655d3 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:13.125342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21879ec0-3ebb-4bc6-aeba-47dfd77655d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:13.127239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21879ec0-3ebb-4bc6-aeba-47dfd77655d3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:13.144218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6635fe-a3ed-47d1-a8fe-a1a5544dc8b5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:13.163827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 75/144] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:50:12 2022] POST /identity/v3/users => generated 283 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:50:13.168084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:13.181329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e79ae-1c52-4699-ab4f-677e256eee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:13.182039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e79ae-1c52-4699-ab4f-677e256eee98 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:13.183111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e79ae-1c52-4699-ab4f-677e256eee98 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:13.183495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e79ae-1c52-4699-ab4f-677e256eee98 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:13.184366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e79ae-1c52-4699-ab4f-677e256eee98 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:13.184801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e79ae-1c52-4699-ab4f-677e256eee98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:13.186326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e79ae-1c52-4699-ab4f-677e256eee98 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:13.195849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 70/145] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:13.202225 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21879ec0-3ebb-4bc6-aeba-47dfd77655d3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:13.346902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21879ec0-3ebb-4bc6-aeba-47dfd77655d3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:13.400306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73c4bc-74b4-49ee-bff3-bf66ceccb420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:13.401602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73c4bc-74b4-49ee-bff3-bf66ceccb420 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:13.402111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73c4bc-74b4-49ee-bff3-bf66ceccb420 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:13.402620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73c4bc-74b4-49ee-bff3-bf66ceccb420 None admin] PATH_INFO: `/v3/users/9a041adb8e4f4e4abce713c554736b14` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:13.404089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73c4bc-74b4-49ee-bff3-bf66ceccb420 None admin] RBAC: Authorizing `identity:get_user(user_id=9a041adb8e4f4e4abce713c554736b14)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:13.405770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73c4bc-74b4-49ee-bff3-bf66ceccb420 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:13.408685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 76/146] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:13 2022] GET /identity/v3/users/9a041adb8e4f4e4abce713c554736b14 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:13.417870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e79ae-1c52-4699-ab4f-677e256eee98 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:13.577742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e79ae-1c52-4699-ab4f-677e256eee98 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:13.630062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9ebe4-0276-4c4e-9e5c-7038b898e749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:13.630679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9ebe4-0276-4c4e-9e5c-7038b898e749 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:13.630970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9ebe4-0276-4c4e-9e5c-7038b898e749 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:13.631252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9ebe4-0276-4c4e-9e5c-7038b898e749 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:13.700627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:13.722815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9ebe4-0276-4c4e-9e5c-7038b898e749 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:13.724337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9ebe4-0276-4c4e-9e5c-7038b898e749 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:13.819318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d290d09-afdd-4e2b-92e5-c7c1cf543d82 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:13.819754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d290d09-afdd-4e2b-92e5-c7c1cf543d82 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:13.819837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d290d09-afdd-4e2b-92e5-c7c1cf543d82 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:13.822236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 77/147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:13.826942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cb182-c34e-4b19-9802-b58881a23c39 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:13.827236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cb182-c34e-4b19-9802-b58881a23c39 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:13.827519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1cb182-c34e-4b19-9802-b58881a23c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:13.840149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:13.849339 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 71/148] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:13.855925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9ebe4-0276-4c4e-9e5c-7038b898e749 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:13.868468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6a1cb182-c34e-4b19-9802-b58881a23c39 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:13.869652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6a1cb182-c34e-4b19-9802-b58881a23c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:13.950928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:13.968725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9ebe4-0276-4c4e-9e5c-7038b898e749 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:14.007562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958eb52-b8b8-4600-b2e4-d106146af05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:14.008448 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958eb52-b8b8-4600-b2e4-d106146af05d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:14.008739 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958eb52-b8b8-4600-b2e4-d106146af05d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:14.009039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c958eb52-b8b8-4600-b2e4-d106146af05d None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:14.010226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958eb52-b8b8-4600-b2e4-d106146af05d None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:14.011738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c958eb52-b8b8-4600-b2e4-d106146af05d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:14.013685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 72/149] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:13 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:14.020494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958eb52-b8b8-4600-b2e4-d106146af05d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:14.052425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 78/150] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:14.093480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cb182-c34e-4b19-9802-b58881a23c39 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:14.199106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c958eb52-b8b8-4600-b2e4-d106146af05d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:14.263429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1cb182-c34e-4b19-9802-b58881a23c39 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:14.279184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb03c8-336a-4a3a-b008-e802c7554a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:14.280122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb03c8-336a-4a3a-b008-e802c7554a1c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:14.280572 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb03c8-336a-4a3a-b008-e802c7554a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:14.281077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb03c8-336a-4a3a-b008-e802c7554a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:14.282066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb03c8-336a-4a3a-b008-e802c7554a1c None admin] RBAC: Adding query filter params (role.id=d3bdf57150744faf9c5c25c317df48fc, scope.project.id=bb614e1d464d4df1959debd82ba5049c, user.id=9a041adb8e4f4e4abce713c554736b14) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:14.282569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb03c8-336a-4a3a-b008-e802c7554a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:14.284182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb03c8-336a-4a3a-b008-e802c7554a1c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:14.293719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 73/151] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:14 2022] GET /identity/v3/role_assignments?user.id=9a041adb8e4f4e4abce713c554736b14&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:14.304504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcfb7a-c0e8-4efa-a65f-7f4e111bdfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:14.305324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcfb7a-c0e8-4efa-a65f-7f4e111bdfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:14.305607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcfb7a-c0e8-4efa-a65f-7f4e111bdfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:14.305884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcfb7a-c0e8-4efa-a65f-7f4e111bdfe5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:14.320242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcfb7a-c0e8-4efa-a65f-7f4e111bdfe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7efcfb7a-c0e8-4efa-a65f-7f4e111bdfe5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:14.328988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:14.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 79/152] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:14.331126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:14.340416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb03c8-336a-4a3a-b008-e802c7554a1c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:14.361567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:14.463193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb03c8-336a-4a3a-b008-e802c7554a1c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:14.503220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bf42b-c224-437c-932a-27e9236b80b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:14.504287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bf42b-c224-437c-932a-27e9236b80b1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:14.504814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bf42b-c224-437c-932a-27e9236b80b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:14.505300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bf42b-c224-437c-932a-27e9236b80b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:14.506290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bf42b-c224-437c-932a-27e9236b80b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:14.506856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bf42b-c224-437c-932a-27e9236b80b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:14.508480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bf42b-c224-437c-932a-27e9236b80b1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:14.522963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 74/153] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:14.529107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcfb7a-c0e8-4efa-a65f-7f4e111bdfe5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:14.697128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcfb7a-c0e8-4efa-a65f-7f4e111bdfe5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:14.748456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13805d2-73f9-4b8d-a53c-3b90ab884159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:14.749369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13805d2-73f9-4b8d-a53c-3b90ab884159 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:14.749655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13805d2-73f9-4b8d-a53c-3b90ab884159 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:14.749937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13805d2-73f9-4b8d-a53c-3b90ab884159 None admin] PATH_INFO: `/v3/users/81b63e4c88d843ffb38c72010e564bba` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:14.751298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13805d2-73f9-4b8d-a53c-3b90ab884159 None admin] RBAC: Authorizing `identity:get_user(user_id=81b63e4c88d843ffb38c72010e564bba)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:14.752811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13805d2-73f9-4b8d-a53c-3b90ab884159 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:14.754861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 80/154] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:14 2022] GET /identity/v3/users/81b63e4c88d843ffb38c72010e564bba => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:14.763555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bf42b-c224-437c-932a-27e9236b80b1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:14.882987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:14.907013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bf42b-c224-437c-932a-27e9236b80b1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:14.943947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0a3e-002d-4d84-853b-137ca5c0bd30 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:14.944445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0a3e-002d-4d84-853b-137ca5c0bd30 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:14.944545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0a3e-002d-4d84-853b-137ca5c0bd30 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:14.948202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 81/155] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:14.953869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87ecd5-9256-4ba1-8b04-84f7a58962ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:14.954778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87ecd5-9256-4ba1-8b04-84f7a58962ce None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:14.955075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87ecd5-9256-4ba1-8b04-84f7a58962ce None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:14.955361 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87ecd5-9256-4ba1-8b04-84f7a58962ce None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:14.956158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695244d-bc4d-40a2-8a02-420b144bdcac None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:14.956458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695244d-bc4d-40a2-8a02-420b144bdcac None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:14.956743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695244d-bc4d-40a2-8a02-420b144bdcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:14.957362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87ecd5-9256-4ba1-8b04-84f7a58962ce None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:14.958893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87ecd5-9256-4ba1-8b04-84f7a58962ce None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:14.960642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 75/156] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:14 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:14.966249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87ecd5-9256-4ba1-8b04-84f7a58962ce None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:14.992578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a695244d-bc4d-40a2-8a02-420b144bdcac None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:14.993792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a695244d-bc4d-40a2-8a02-420b144bdcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:15.079776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:15.093381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 82/157] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:15.135319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87ecd5-9256-4ba1-8b04-84f7a58962ce None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:15.139788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695244d-bc4d-40a2-8a02-420b144bdcac None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:15.171205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0bb1d-15b7-4226-ab96-3ed1b8a47206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:15.172159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0bb1d-15b7-4226-ab96-3ed1b8a47206 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:15.172445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0bb1d-15b7-4226-ab96-3ed1b8a47206 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:15.172897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0bb1d-15b7-4226-ab96-3ed1b8a47206 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c/users/81b63e4c88d843ffb38c72010e564bba/roles/23aa53c14bd94d60b9e3ffdcfbecd8dc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:15.175062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0bb1d-15b7-4226-ab96-3ed1b8a47206 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb614e1d464d4df1959debd82ba5049c, user_id=81b63e4c88d843ffb38c72010e564bba, role_id=23aa53c14bd94d60b9e3ffdcfbecd8dc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:15.176663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0bb1d-15b7-4226-ab96-3ed1b8a47206 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:15.195103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 76/158] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:50:14 2022] PUT /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c/users/81b63e4c88d843ffb38c72010e564bba/roles/23aa53c14bd94d60b9e3ffdcfbecd8dc => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:50:15.288484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695244d-bc4d-40a2-8a02-420b144bdcac None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:15.329954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878e7ed-dc06-473f-a234-f73909945b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:15.331018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878e7ed-dc06-473f-a234-f73909945b26 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:15.331446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878e7ed-dc06-473f-a234-f73909945b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:15.331900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878e7ed-dc06-473f-a234-f73909945b26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:15.347148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878e7ed-dc06-473f-a234-f73909945b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4878e7ed-dc06-473f-a234-f73909945b26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:15.356117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:15.357247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:15.358218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:15.360364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 83/159] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:50:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:15.366705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0bb1d-15b7-4226-ab96-3ed1b8a47206 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:15.377194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:15.485421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0bb1d-15b7-4226-ab96-3ed1b8a47206 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:15.521837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b237d-eb08-4e3c-a46f-2a5e4d785961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:15.522562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b237d-eb08-4e3c-a46f-2a5e4d785961 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:15.522851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b237d-eb08-4e3c-a46f-2a5e4d785961 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:15.523133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b237d-eb08-4e3c-a46f-2a5e4d785961 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:15.523902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b237d-eb08-4e3c-a46f-2a5e4d785961 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:15.524214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b237d-eb08-4e3c-a46f-2a5e4d785961 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:15.525727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b237d-eb08-4e3c-a46f-2a5e4d785961 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:15.535659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801e4fe-29a4-4d0c-b517-61653ca89522 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:15.535956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801e4fe-29a4-4d0c-b517-61653ca89522 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:15.536224 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801e4fe-29a4-4d0c-b517-61653ca89522 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:15.537661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 84/160] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:50:15.539854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 77/161] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:15.547588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801e4fe-29a4-4d0c-b517-61653ca89522 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:15.550469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca22ded-e8b1-4cd4-b595-a469bbeb8d70 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:15.550469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca22ded-e8b1-4cd4-b595-a469bbeb8d70 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:15.550665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca22ded-e8b1-4cd4-b595-a469bbeb8d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:15.582637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-aca22ded-e8b1-4cd4-b595-a469bbeb8d70 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:15.582637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-aca22ded-e8b1-4cd4-b595-a469bbeb8d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:15.643731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:15.677585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:15.683777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 78/162] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:15.717356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801e4fe-29a4-4d0c-b517-61653ca89522 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:15.736701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca22ded-e8b1-4cd4-b595-a469bbeb8d70 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:15.783760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e95146-7f93-4894-a366-98afdf2fe524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:15.784645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e95146-7f93-4894-a366-98afdf2fe524 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:15.784932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e95146-7f93-4894-a366-98afdf2fe524 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:15.785432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e95146-7f93-4894-a366-98afdf2fe524 None admin] PATH_INFO: `/v3/users/c2ba123046b04572883d482380082522` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:15.786411 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e95146-7f93-4894-a366-98afdf2fe524 None admin] RBAC: Authorizing `identity:get_user(user_id=c2ba123046b04572883d482380082522)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:15.787936 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e95146-7f93-4894-a366-98afdf2fe524 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:15.790237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 85/163] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:15 2022] GET /identity/v3/users/c2ba123046b04572883d482380082522 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:15.799838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e95146-7f93-4894-a366-98afdf2fe524 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:15.893500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca22ded-e8b1-4cd4-b595-a469bbeb8d70 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:15.936167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06aede-e4ec-44cd-bd52-4ed07b43f3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:15.938140 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06aede-e4ec-44cd-bd52-4ed07b43f3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:15.938505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06aede-e4ec-44cd-bd52-4ed07b43f3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:15.938789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06aede-e4ec-44cd-bd52-4ed07b43f3b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:15.963999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06aede-e4ec-44cd-bd52-4ed07b43f3b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7f06aede-e4ec-44cd-bd52-4ed07b43f3b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:15.983335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:15.984261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:15.985045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 79/164] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:15.992077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06aede-e4ec-44cd-bd52-4ed07b43f3b7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:15.999571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e95146-7f93-4894-a366-98afdf2fe524 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:16.054409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:16.067153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:16.103678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06aede-e4ec-44cd-bd52-4ed07b43f3b7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:16.107159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d70408-73f2-4887-8679-73872be7c926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:16.108048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d70408-73f2-4887-8679-73872be7c926 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:16.108487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d70408-73f2-4887-8679-73872be7c926 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:16.108915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d70408-73f2-4887-8679-73872be7c926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:16.139394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49921-3fdf-4ce5-bee7-23a25912a025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:16.140341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49921-3fdf-4ce5-bee7-23a25912a025 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:16.140772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49921-3fdf-4ce5-bee7-23a25912a025 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:16.141193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49921-3fdf-4ce5-bee7-23a25912a025 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:16.142083 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49921-3fdf-4ce5-bee7-23a25912a025 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:16.142565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49921-3fdf-4ce5-bee7-23a25912a025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:16.144156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49921-3fdf-4ce5-bee7-23a25912a025 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:16.153490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 80/165] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:16.155175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426851d-a33b-46b6-b813-266f2a9b1e25 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:16.155672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426851d-a33b-46b6-b813-266f2a9b1e25 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:16.156128 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426851d-a33b-46b6-b813-266f2a9b1e25 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:16.157539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 81/166] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:16.159267 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426851d-a33b-46b6-b813-266f2a9b1e25 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:16.212125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d70408-73f2-4887-8679-73872be7c926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:16.214202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d70408-73f2-4887-8679-73872be7c926 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:16.269915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426851d-a33b-46b6-b813-266f2a9b1e25 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:16.307295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332a33cf-d220-4a08-b3ca-e163c921ee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:16.308551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a33cf-d220-4a08-b3ca-e163c921ee05 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:16.309025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a33cf-d220-4a08-b3ca-e163c921ee05 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:16.309487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332a33cf-d220-4a08-b3ca-e163c921ee05 None admin] PATH_INFO: `/v3/users/9a041adb8e4f4e4abce713c554736b14` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:16.311369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a33cf-d220-4a08-b3ca-e163c921ee05 None admin] RBAC: Authorizing `identity:get_user(user_id=9a041adb8e4f4e4abce713c554736b14)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:16.313216 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332a33cf-d220-4a08-b3ca-e163c921ee05 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:16.315762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 82/167] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:16 2022] GET /identity/v3/users/9a041adb8e4f4e4abce713c554736b14 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:16.317358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0c25f-f8ab-41c0-bf71-eae59be36bd9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:16.317828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0c25f-f8ab-41c0-bf71-eae59be36bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:16.318285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0c25f-f8ab-41c0-bf71-eae59be36bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:16.326835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:16.352510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1ed0c25f-f8ab-41c0-bf71-eae59be36bd9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:16.355612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1ed0c25f-f8ab-41c0-bf71-eae59be36bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:16.405118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 86/168] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:16.407745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d70408-73f2-4887-8679-73872be7c926 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:16.443782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:16.465327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 83/169] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:16.466624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0c25f-f8ab-41c0-bf71-eae59be36bd9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:16.571737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0c25f-f8ab-41c0-bf71-eae59be36bd9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:16.581930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d70408-73f2-4887-8679-73872be7c926 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:16.598828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:16.622799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825eaf0-b1f2-4bd6-ba32-243d0c2c68f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:16.624090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825eaf0-b1f2-4bd6-ba32-243d0c2c68f9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:16.624803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825eaf0-b1f2-4bd6-ba32-243d0c2c68f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:16.625891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825eaf0-b1f2-4bd6-ba32-243d0c2c68f9 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:16.628499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825eaf0-b1f2-4bd6-ba32-243d0c2c68f9 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:16.630920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825eaf0-b1f2-4bd6-ba32-243d0c2c68f9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:16.633433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 84/170] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:16 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:16.636387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825eaf0-b1f2-4bd6-ba32-243d0c2c68f9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:16.652572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ffeb7-b869-4a78-98c9-4bf66c1972e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:16.653238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ffeb7-b869-4a78-98c9-4bf66c1972e1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:16.653526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ffeb7-b869-4a78-98c9-4bf66c1972e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:16.653807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ffeb7-b869-4a78-98c9-4bf66c1972e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:16.746834 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825eaf0-b1f2-4bd6-ba32-243d0c2c68f9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:16.786792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d67c3-47fe-47b0-85d6-4ba3c7142ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:16.787961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d67c3-47fe-47b0-85d6-4ba3c7142ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:16.788570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d67c3-47fe-47b0-85d6-4ba3c7142ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:16.789182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d67c3-47fe-47b0-85d6-4ba3c7142ee6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:16.800734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5d67c3-47fe-47b0-85d6-4ba3c7142ee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0b5d67c3-47fe-47b0-85d6-4ba3c7142ee6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:16.809757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:16.810967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:16.811747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:16.814646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 85/171] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:16.815676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d67c3-47fe-47b0-85d6-4ba3c7142ee6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:16.839682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ffeb7-b869-4a78-98c9-4bf66c1972e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:16.841315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ffeb7-b869-4a78-98c9-4bf66c1972e1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:16.865711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:16.878942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:16.928186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d67c3-47fe-47b0-85d6-4ba3c7142ee6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:16.965578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba02b2a-1d61-4a30-8b1d-326cf63e892a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:16.966657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba02b2a-1d61-4a30-8b1d-326cf63e892a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:16.967142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba02b2a-1d61-4a30-8b1d-326cf63e892a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:16.967639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba02b2a-1d61-4a30-8b1d-326cf63e892a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:16.968672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba02b2a-1d61-4a30-8b1d-326cf63e892a None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=bb614e1d464d4df1959debd82ba5049c, user.id=c2ba123046b04572883d482380082522) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:16.969198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba02b2a-1d61-4a30-8b1d-326cf63e892a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:16.970976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba02b2a-1d61-4a30-8b1d-326cf63e892a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:16.977606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 86/172] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:16 2022] GET /identity/v3/role_assignments?user.id=c2ba123046b04572883d482380082522&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:16.978722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba02b2a-1d61-4a30-8b1d-326cf63e892a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:17.035175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 87/173] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:17.040795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ffeb7-b869-4a78-98c9-4bf66c1972e1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:17.094717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba02b2a-1d61-4a30-8b1d-326cf63e892a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:17.110812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:17.160485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cc1eb-3f01-4e6c-94d7-7925f0ed67d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:17.163297 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cc1eb-3f01-4e6c-94d7-7925f0ed67d8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:17.164051 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cc1eb-3f01-4e6c-94d7-7925f0ed67d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:17.165695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cc1eb-3f01-4e6c-94d7-7925f0ed67d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:17.167264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cc1eb-3f01-4e6c-94d7-7925f0ed67d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:17.168037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cc1eb-3f01-4e6c-94d7-7925f0ed67d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:17.171176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27cc1eb-3f01-4e6c-94d7-7925f0ed67d8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:17.189563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 87/174] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:17.195628 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cc1eb-3f01-4e6c-94d7-7925f0ed67d8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:17.252998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ffeb7-b869-4a78-98c9-4bf66c1972e1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:17.279071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:17.303350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c19ec4-232d-4a22-bede-76b7df321378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:17.303350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19ec4-232d-4a22-bede-76b7df321378 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:17.303350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19ec4-232d-4a22-bede-76b7df321378 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:17.303863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19ec4-232d-4a22-bede-76b7df321378 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:17.305443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c19ec4-232d-4a22-bede-76b7df321378 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:17.307931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c19ec4-232d-4a22-bede-76b7df321378 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:17.310497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 88/175] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:17 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:17.316710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c19ec4-232d-4a22-bede-76b7df321378 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:17.395011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cc1eb-3f01-4e6c-94d7-7925f0ed67d8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:17.439000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c19ec4-232d-4a22-bede-76b7df321378 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:17.458748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1be1f-26d0-4dd9-bafb-884263835cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:17.459867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1be1f-26d0-4dd9-bafb-884263835cdc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:17.460454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1be1f-26d0-4dd9-bafb-884263835cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:17.460876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1be1f-26d0-4dd9-bafb-884263835cdc None admin] PATH_INFO: `/v3/users/9a041adb8e4f4e4abce713c554736b14` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:17.462235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1be1f-26d0-4dd9-bafb-884263835cdc None admin] RBAC: Authorizing `identity:get_user(user_id=9a041adb8e4f4e4abce713c554736b14)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:17.464600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1be1f-26d0-4dd9-bafb-884263835cdc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:17.470639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 88/176] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:17 2022] GET /identity/v3/users/9a041adb8e4f4e4abce713c554736b14 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:17.472187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1be1f-26d0-4dd9-bafb-884263835cdc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:17.527608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66175d-af36-4135-b679-cb4366ca75f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:17.528269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66175d-af36-4135-b679-cb4366ca75f9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:17.528560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66175d-af36-4135-b679-cb4366ca75f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:17.528841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66175d-af36-4135-b679-cb4366ca75f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:17.529700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66175d-af36-4135-b679-cb4366ca75f9 None admin] RBAC: Adding query filter params (role.id=fa67cb2f211d433097f606ff029f863e, scope.project.id=bb614e1d464d4df1959debd82ba5049c, user.id=9a041adb8e4f4e4abce713c554736b14) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:17.530021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66175d-af36-4135-b679-cb4366ca75f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:17.531532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66175d-af36-4135-b679-cb4366ca75f9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:17.544326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:17.545107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 89/177] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:17 2022] GET /identity/v3/role_assignments?user.id=9a041adb8e4f4e4abce713c554736b14&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=fa67cb2f211d433097f606ff029f863e => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:17.627962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1be1f-26d0-4dd9-bafb-884263835cdc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:17.677447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a2ac3-3757-4bae-af07-9892187f9f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:17.678120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a2ac3-3757-4bae-af07-9892187f9f26 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:17.678411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a2ac3-3757-4bae-af07-9892187f9f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:17.678723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a2ac3-3757-4bae-af07-9892187f9f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:17.778379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a2ac3-3757-4bae-af07-9892187f9f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:17.782479 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a2ac3-3757-4bae-af07-9892187f9f26 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:17.871762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:17.946144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 89/178] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:50:17.951689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66175d-af36-4135-b679-cb4366ca75f9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:18.017050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:18.083398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66175d-af36-4135-b679-cb4366ca75f9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:18.133364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec2f14-a094-46ad-9f6d-5015a91bf761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:18.134241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec2f14-a094-46ad-9f6d-5015a91bf761 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.134557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec2f14-a094-46ad-9f6d-5015a91bf761 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.134841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec2f14-a094-46ad-9f6d-5015a91bf761 None admin] PATH_INFO: `/v3/projects/3ffcc0e7c1584efaa5fa4473241a9bf6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.139197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec2f14-a094-46ad-9f6d-5015a91bf761 None admin] RBAC: Authorizing `identity:get_project(project_id=3ffcc0e7c1584efaa5fa4473241a9bf6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:18.140702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec2f14-a094-46ad-9f6d-5015a91bf761 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:18.142474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 90/179] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:17 2022] GET /identity/v3/projects/3ffcc0e7c1584efaa5fa4473241a9bf6 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:18.148433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a2ac3-3757-4bae-af07-9892187f9f26 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:18.267884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a2ac3-3757-4bae-af07-9892187f9f26 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:18.306605 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86045fe-ed47-4fab-b517-5d73bec526bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:18.307322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86045fe-ed47-4fab-b517-5d73bec526bb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.307629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86045fe-ed47-4fab-b517-5d73bec526bb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.307893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86045fe-ed47-4fab-b517-5d73bec526bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.308744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86045fe-ed47-4fab-b517-5d73bec526bb None admin] RBAC: Adding query filter params (role.id=d3bdf57150744faf9c5c25c317df48fc, scope.project.id=3ffcc0e7c1584efaa5fa4473241a9bf6, user.id=9a041adb8e4f4e4abce713c554736b14) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:18.309060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86045fe-ed47-4fab-b517-5d73bec526bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:18.310532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86045fe-ed47-4fab-b517-5d73bec526bb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:18.317261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 90/180] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:18 2022] GET /identity/v3/role_assignments?user.id=9a041adb8e4f4e4abce713c554736b14&scope.project.id=3ffcc0e7c1584efaa5fa4473241a9bf6&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:18.391992 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ca305-119e-44f0-9240-1ec2c853c6da None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.392320 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ca305-119e-44f0-9240-1ec2c853c6da None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.392573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ca305-119e-44f0-9240-1ec2c853c6da None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.394008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 91/181] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:18.399804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0548c-3408-4113-a486-3d5b15e4e82c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.400314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0548c-3408-4113-a486-3d5b15e4e82c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.400741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0548c-3408-4113-a486-3d5b15e4e82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.407812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:18.423644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f6c0548c-3408-4113-a486-3d5b15e4e82c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:18.424881 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f6c0548c-3408-4113-a486-3d5b15e4e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:18.521147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 91/182] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:18.542509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9e699-35d6-4425-8f0c-a6c1a38497cd None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.543137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9e699-35d6-4425-8f0c-a6c1a38497cd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.543350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9e699-35d6-4425-8f0c-a6c1a38497cd None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.544780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 92/183] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:18.555789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fe2cc-cbaf-4307-91c2-78610079c575 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.556348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fe2cc-cbaf-4307-91c2-78610079c575 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.556670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fe2cc-cbaf-4307-91c2-78610079c575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.569431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9e699-35d6-4425-8f0c-a6c1a38497cd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:18.584868 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ba6fe2cc-cbaf-4307-91c2-78610079c575 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:18.586085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ba6fe2cc-cbaf-4307-91c2-78610079c575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:18.689711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 92/184] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:18.691517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18146d-f9ab-42ec-a942-2f1b8ddb9c59 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.691812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18146d-f9ab-42ec-a942-2f1b8ddb9c59 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.692094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae18146d-f9ab-42ec-a942-2f1b8ddb9c59 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.694539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 93/185] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:18.700288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdd795-b5b4-47c3-92a6-38ec12ad8e07 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.700581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdd795-b5b4-47c3-92a6-38ec12ad8e07 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.700993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdd795-b5b4-47c3-92a6-38ec12ad8e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.721057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9e699-35d6-4425-8f0c-a6c1a38497cd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:18.722870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f1cdd795-b5b4-47c3-92a6-38ec12ad8e07 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:18.724065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f1cdd795-b5b4-47c3-92a6-38ec12ad8e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:18.747158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:18.776902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f602827-0a5d-4109-8cb7-a0c894399d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:18.777632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f602827-0a5d-4109-8cb7-a0c894399d63 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.777924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f602827-0a5d-4109-8cb7-a0c894399d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.778476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f602827-0a5d-4109-8cb7-a0c894399d63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.779434 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f602827-0a5d-4109-8cb7-a0c894399d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:18.780922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f602827-0a5d-4109-8cb7-a0c894399d63 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:18.786917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 93/186] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:18.786917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f602827-0a5d-4109-8cb7-a0c894399d63 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:18.802257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:18.820957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 94/187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:18.823486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdd795-b5b4-47c3-92a6-38ec12ad8e07 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:18.897751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f602827-0a5d-4109-8cb7-a0c894399d63 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:18.935185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc765c7-d4fa-4701-bccf-ca08892838d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:18.935972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc765c7-d4fa-4701-bccf-ca08892838d9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.936258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc765c7-d4fa-4701-bccf-ca08892838d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.936536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc765c7-d4fa-4701-bccf-ca08892838d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.947062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc765c7-d4fa-4701-bccf-ca08892838d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:18.947214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdd795-b5b4-47c3-92a6-38ec12ad8e07 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5bc765c7-d4fa-4701-bccf-ca08892838d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:18.955458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:18.956343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 94/188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:50:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:18.957557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc765c7-d4fa-4701-bccf-ca08892838d9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:18.990155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4e1b9-d71d-4ef7-9390-3693a42b0bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:18.991123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4e1b9-d71d-4ef7-9390-3693a42b0bbc None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:18.991410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4e1b9-d71d-4ef7-9390-3693a42b0bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:18.991688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4e1b9-d71d-4ef7-9390-3693a42b0bbc None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:18.992393 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4e1b9-d71d-4ef7-9390-3693a42b0bbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:18.995347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4e1b9-d71d-4ef7-9390-3693a42b0bbc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:19.007962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:19.023070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 95/189] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:50:18 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:19.031693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4e1b9-d71d-4ef7-9390-3693a42b0bbc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:19.066728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc765c7-d4fa-4701-bccf-ca08892838d9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:19.098059 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:19.111356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546696c2-07da-4c3c-858d-9eac3ed643ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:19.112838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546696c2-07da-4c3c-858d-9eac3ed643ed None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:19.113287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546696c2-07da-4c3c-858d-9eac3ed643ed None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:19.113586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546696c2-07da-4c3c-858d-9eac3ed643ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:19.126057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546696c2-07da-4c3c-858d-9eac3ed643ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-546696c2-07da-4c3c-858d-9eac3ed643ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:19.134602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:19.136232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:19.137036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:19.138810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 95/190] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:19.140582 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2d175-b513-477c-a5b4-c9283c1f9e52 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:19.141255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2d175-b513-477c-a5b4-c9283c1f9e52 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:19.141674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2d175-b513-477c-a5b4-c9283c1f9e52 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:19.143719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 96/191] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:19.144313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2d175-b513-477c-a5b4-c9283c1f9e52 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:19.163593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4e1b9-d71d-4ef7-9390-3693a42b0bbc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:19.204121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d3910-03d3-4264-a284-23575b13a30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:19.204917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d3910-03d3-4264-a284-23575b13a30c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:19.205208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d3910-03d3-4264-a284-23575b13a30c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:19.205896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d3910-03d3-4264-a284-23575b13a30c None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:19.206290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d3910-03d3-4264-a284-23575b13a30c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:19.206631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d3910-03d3-4264-a284-23575b13a30c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:19.208026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d3910-03d3-4264-a284-23575b13a30c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:19.217590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 96/192] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:19.219647 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b6d0-4d72-4d07-8669-5dc2fd7efac1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:19.219950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b6d0-4d72-4d07-8669-5dc2fd7efac1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:19.220237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b6d0-4d72-4d07-8669-5dc2fd7efac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:19.238900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-06b9b6d0-4d72-4d07-8669-5dc2fd7efac1 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:19.240165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-06b9b6d0-4d72-4d07-8669-5dc2fd7efac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:19.259962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2d175-b513-477c-a5b4-c9283c1f9e52 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:19.269493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:19.298821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33ac7b-cc4a-416f-a4d2-815624ad403d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:19.299495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33ac7b-cc4a-416f-a4d2-815624ad403d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:19.299774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33ac7b-cc4a-416f-a4d2-815624ad403d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:19.300050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33ac7b-cc4a-416f-a4d2-815624ad403d None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:19.300804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33ac7b-cc4a-416f-a4d2-815624ad403d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:19.301114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33ac7b-cc4a-416f-a4d2-815624ad403d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:19.302850 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df33ac7b-cc4a-416f-a4d2-815624ad403d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:19.313060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 97/193] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:19.314099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33ac7b-cc4a-416f-a4d2-815624ad403d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:19.317839 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 97/194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:19.319422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9b6d0-4d72-4d07-8669-5dc2fd7efac1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:19.383098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:19.448663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33ac7b-cc4a-416f-a4d2-815624ad403d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:19.455143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9b6d0-4d72-4d07-8669-5dc2fd7efac1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:19.492728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2506d-9f72-4ed9-8b7a-9cf9a0861748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:19.494027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2506d-9f72-4ed9-8b7a-9cf9a0861748 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:19.494645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2506d-9f72-4ed9-8b7a-9cf9a0861748 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:19.495688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2506d-9f72-4ed9-8b7a-9cf9a0861748 None admin] PATH_INFO: `/v3/users/a009d22d7fc84a9c95fb35ac1635b02c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:19.497318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e53844-b28e-4682-8c97-b17fa395301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:19.498244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e53844-b28e-4682-8c97-b17fa395301a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:19.498742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e53844-b28e-4682-8c97-b17fa395301a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:19.499019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e53844-b28e-4682-8c97-b17fa395301a None admin] PATH_INFO: `/v3/users/a009d22d7fc84a9c95fb35ac1635b02c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:19.514241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2506d-9f72-4ed9-8b7a-9cf9a0861748 None admin] RBAC: Authorizing `identity:get_user(user_id=a009d22d7fc84a9c95fb35ac1635b02c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:19.516161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e53844-b28e-4682-8c97-b17fa395301a None admin] RBAC: Authorizing `identity:get_user(user_id=a009d22d7fc84a9c95fb35ac1635b02c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:19.516583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2506d-9f72-4ed9-8b7a-9cf9a0861748 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:19.517625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e53844-b28e-4682-8c97-b17fa395301a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:19.518885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 98/195] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:19 2022] GET /identity/v3/users/a009d22d7fc84a9c95fb35ac1635b02c => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:19.519374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 98/196] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:19 2022] GET /identity/v3/users/a009d22d7fc84a9c95fb35ac1635b02c => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:19.520187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2506d-9f72-4ed9-8b7a-9cf9a0861748 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:19.531454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e53844-b28e-4682-8c97-b17fa395301a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:19.560849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:19.648306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2506d-9f72-4ed9-8b7a-9cf9a0861748 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:19.652438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:19.655679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e53844-b28e-4682-8c97-b17fa395301a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:19.688149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bee2e-926e-44fd-895d-cc432fa04993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:19.689230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bee2e-926e-44fd-895d-cc432fa04993 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:19.689522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bee2e-926e-44fd-895d-cc432fa04993 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:19.689807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bee2e-926e-44fd-895d-cc432fa04993 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:19.699940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682c54f-a876-45a5-8498-cec6f638480f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:19.700758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682c54f-a876-45a5-8498-cec6f638480f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:19.701109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682c54f-a876-45a5-8498-cec6f638480f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:19.701506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682c54f-a876-45a5-8498-cec6f638480f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:19.703346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bee2e-926e-44fd-895d-cc432fa04993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-011bee2e-926e-44fd-895d-cc432fa04993 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:19.713279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:19.714206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:19.715066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 99/197] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:19.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bee2e-926e-44fd-895d-cc432fa04993 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:19.782946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682c54f-a876-45a5-8498-cec6f638480f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:19.784565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682c54f-a876-45a5-8498-cec6f638480f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:19.838865 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:19.854774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bee2e-926e-44fd-895d-cc432fa04993 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:19.894712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbecf7-d664-4a2f-907a-e534f672da59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:19.895363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbecf7-d664-4a2f-907a-e534f672da59 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:19.895647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbecf7-d664-4a2f-907a-e534f672da59 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:19.895930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbecf7-d664-4a2f-907a-e534f672da59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:19.910933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:19.917270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 99/198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:19.918570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682c54f-a876-45a5-8498-cec6f638480f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:19.970015 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbecf7-d664-4a2f-907a-e534f672da59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:19.971822 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbecf7-d664-4a2f-907a-e534f672da59 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:20.040334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682c54f-a876-45a5-8498-cec6f638480f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:20.068143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:20.078803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805f03-31bd-4b08-bb51-4f7626d864d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:20.079760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805f03-31bd-4b08-bb51-4f7626d864d2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:20.080240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805f03-31bd-4b08-bb51-4f7626d864d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:20.080729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f805f03-31bd-4b08-bb51-4f7626d864d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:20.081654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f805f03-31bd-4b08-bb51-4f7626d864d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:20.082163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f805f03-31bd-4b08-bb51-4f7626d864d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:20.082531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 100/199] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:20.083499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbecf7-d664-4a2f-907a-e534f672da59 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:20.083965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f805f03-31bd-4b08-bb51-4f7626d864d2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:20.092730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 100/200] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:20.093778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805f03-31bd-4b08-bb51-4f7626d864d2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:20.158891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:20.197677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbecf7-d664-4a2f-907a-e534f672da59 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:20.200833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f805f03-31bd-4b08-bb51-4f7626d864d2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:20.235047 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99fb19-4972-4cfd-94c4-36c101f3ec70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:20.235380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb41b4-8786-4dec-83ef-01dc182c6fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:20.236395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99fb19-4972-4cfd-94c4-36c101f3ec70 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:20.236502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb41b4-8786-4dec-83ef-01dc182c6fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:20.236685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb41b4-8786-4dec-83ef-01dc182c6fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:20.236970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb41b4-8786-4dec-83ef-01dc182c6fd6 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:20.237367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99fb19-4972-4cfd-94c4-36c101f3ec70 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:20.237829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99fb19-4972-4cfd-94c4-36c101f3ec70 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:20.238171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb41b4-8786-4dec-83ef-01dc182c6fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:20.239420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99fb19-4972-4cfd-94c4-36c101f3ec70 None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:20.239680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb41b4-8786-4dec-83ef-01dc182c6fd6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:20.241260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99fb19-4972-4cfd-94c4-36c101f3ec70 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:20.241689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 101/201] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:20 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:20.242483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb41b4-8786-4dec-83ef-01dc182c6fd6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:20.243804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 101/202] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:20 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:20.254882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99fb19-4972-4cfd-94c4-36c101f3ec70 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:20.303580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:20.351498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb41b4-8786-4dec-83ef-01dc182c6fd6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:20.368912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99fb19-4972-4cfd-94c4-36c101f3ec70 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:20.391765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5b17d-d028-4b62-b798-5e39ba086258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:20.392987 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5b17d-d028-4b62-b798-5e39ba086258 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:20.393485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5b17d-d028-4b62-b798-5e39ba086258 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:20.393947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5b17d-d028-4b62-b798-5e39ba086258 None admin] PATH_INFO: `/v3/users/81b63e4c88d843ffb38c72010e564bba` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:20.395377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5b17d-d028-4b62-b798-5e39ba086258 None admin] RBAC: Authorizing `identity:get_user(user_id=81b63e4c88d843ffb38c72010e564bba)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:20.397243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5b17d-d028-4b62-b798-5e39ba086258 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:20.403761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 102/203] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:20 2022] GET /identity/v3/users/81b63e4c88d843ffb38c72010e564bba => generated 289 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:20.404652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:20.405804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5b17d-d028-4b62-b798-5e39ba086258 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:20.408112 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed72af-81ad-4d7c-b9ad-0f2f244c1127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:20.408900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed72af-81ad-4d7c-b9ad-0f2f244c1127 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:20.409338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed72af-81ad-4d7c-b9ad-0f2f244c1127 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:20.410510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed72af-81ad-4d7c-b9ad-0f2f244c1127 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:20.410734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed72af-81ad-4d7c-b9ad-0f2f244c1127 None admin] RBAC: Adding query filter params (role.id=fa67cb2f211d433097f606ff029f863e, scope.project.id=42dcbe790c2a48f2bd52298af1b58573, user.id=a009d22d7fc84a9c95fb35ac1635b02c) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:20.411119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed72af-81ad-4d7c-b9ad-0f2f244c1127 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:20.412775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed72af-81ad-4d7c-b9ad-0f2f244c1127 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:20.420091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 102/204] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:20 2022] GET /identity/v3/role_assignments?user.id=a009d22d7fc84a9c95fb35ac1635b02c&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=fa67cb2f211d433097f606ff029f863e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:20.420886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed72af-81ad-4d7c-b9ad-0f2f244c1127 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:20.521354 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5b17d-d028-4b62-b798-5e39ba086258 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:20.537704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed72af-81ad-4d7c-b9ad-0f2f244c1127 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:20.549929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:20.562286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718903b1-e3d4-4517-bd6c-0121752b9e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:20.563102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718903b1-e3d4-4517-bd6c-0121752b9e3d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:20.563482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718903b1-e3d4-4517-bd6c-0121752b9e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:20.563837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718903b1-e3d4-4517-bd6c-0121752b9e3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:20.564756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718903b1-e3d4-4517-bd6c-0121752b9e3d None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=42dcbe790c2a48f2bd52298af1b58573, user.id=a009d22d7fc84a9c95fb35ac1635b02c) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:20.565179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718903b1-e3d4-4517-bd6c-0121752b9e3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:20.566773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718903b1-e3d4-4517-bd6c-0121752b9e3d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:20.573795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 103/205] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:20 2022] GET /identity/v3/role_assignments?user.id=a009d22d7fc84a9c95fb35ac1635b02c&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:20.580896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe8d74-9710-4f91-ae9f-031a05742f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:20.581585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe8d74-9710-4f91-ae9f-031a05742f94 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:20.581873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe8d74-9710-4f91-ae9f-031a05742f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:20.582148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe8d74-9710-4f91-ae9f-031a05742f94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:20.657985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:20.667925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe8d74-9710-4f91-ae9f-031a05742f94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:20.669469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe8d74-9710-4f91-ae9f-031a05742f94 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:20.784403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 103/206] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:20.789942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718903b1-e3d4-4517-bd6c-0121752b9e3d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:20.935043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718903b1-e3d4-4517-bd6c-0121752b9e3d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:20.969169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7668bb-3942-4c71-9c08-72c463f6077b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:20.970047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7668bb-3942-4c71-9c08-72c463f6077b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:20.970695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7668bb-3942-4c71-9c08-72c463f6077b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:20.970983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7668bb-3942-4c71-9c08-72c463f6077b None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:20.972217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7668bb-3942-4c71-9c08-72c463f6077b None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:20.973708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7668bb-3942-4c71-9c08-72c463f6077b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:20.975780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 104/207] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:20 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:20.985474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe8d74-9710-4f91-ae9f-031a05742f94 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:21.114834 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:21.121770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe8d74-9710-4f91-ae9f-031a05742f94 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:21.162794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cda24-22c7-4f2d-be87-8a564c69c0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:21.163694 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cda24-22c7-4f2d-be87-8a564c69c0ec None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:21.163981 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cda24-22c7-4f2d-be87-8a564c69c0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:21.164261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cda24-22c7-4f2d-be87-8a564c69c0ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:21.165101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cda24-22c7-4f2d-be87-8a564c69c0ec None admin] RBAC: Adding query filter params (role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc, scope.project.id=bb614e1d464d4df1959debd82ba5049c, user.id=81b63e4c88d843ffb38c72010e564bba) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:21.165415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cda24-22c7-4f2d-be87-8a564c69c0ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:21.167548 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cda24-22c7-4f2d-be87-8a564c69c0ec None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:21.181009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 104/208] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:20 2022] GET /identity/v3/role_assignments?user.id=81b63e4c88d843ffb38c72010e564bba&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:21.562271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a2e8-f759-459d-a19a-7c25ec2b6b49 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:21.563329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a2e8-f759-459d-a19a-7c25ec2b6b49 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:21.563980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a2e8-f759-459d-a19a-7c25ec2b6b49 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:21.565592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 105/209] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:21.571669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7b6dc-09c3-4e43-b972-4b34720822b5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:21.571966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7b6dc-09c3-4e43-b972-4b34720822b5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:21.572236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7b6dc-09c3-4e43-b972-4b34720822b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:21.591817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f8d7b6dc-09c3-4e43-b972-4b34720822b5 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:21.593040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f8d7b6dc-09c3-4e43-b972-4b34720822b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:21.667645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 105/210] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:21.691568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022a2e8-f759-459d-a19a-7c25ec2b6b49 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:21.754585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:21.798286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829aa05-a9c3-446d-ac6e-84fb4ac00e47 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:21.798626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829aa05-a9c3-446d-ac6e-84fb4ac00e47 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:21.798917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829aa05-a9c3-446d-ac6e-84fb4ac00e47 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:21.800449 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 106/211] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:21.805437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dccc2-24d5-4fbf-8c80-b38eb10bdf10 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:21.805726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dccc2-24d5-4fbf-8c80-b38eb10bdf10 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:21.806000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dccc2-24d5-4fbf-8c80-b38eb10bdf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:21.815046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022a2e8-f759-459d-a19a-7c25ec2b6b49 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:21.823670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ec6dccc2-24d5-4fbf-8c80-b38eb10bdf10 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:21.824850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ec6dccc2-24d5-4fbf-8c80-b38eb10bdf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:21.862497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2868e7-bc95-4500-8009-53ba8555daf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:21.863515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2868e7-bc95-4500-8009-53ba8555daf7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:21.863940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2868e7-bc95-4500-8009-53ba8555daf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:21.864500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2868e7-bc95-4500-8009-53ba8555daf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:21.876061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2868e7-bc95-4500-8009-53ba8555daf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-aa2868e7-bc95-4500-8009-53ba8555daf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:21.884266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:21.885383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:21.886361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:21.888442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 106/212] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:21.894075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2868e7-bc95-4500-8009-53ba8555daf7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:21.913030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 107/213] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:21.937493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dccc2-24d5-4fbf-8c80-b38eb10bdf10 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:21.956402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:22.003027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2868e7-bc95-4500-8009-53ba8555daf7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:22.021134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:22.041105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7c091-d382-4da5-85de-29b7f621a162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:22.042487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c091-d382-4da5-85de-29b7f621a162 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:22.043031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c091-d382-4da5-85de-29b7f621a162 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:22.043508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7c091-d382-4da5-85de-29b7f621a162 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:22.044564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7c091-d382-4da5-85de-29b7f621a162 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:22.045085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7c091-d382-4da5-85de-29b7f621a162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:22.046343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dccc2-24d5-4fbf-8c80-b38eb10bdf10 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:22.047053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7c091-d382-4da5-85de-29b7f621a162 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:22.056302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 107/214] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:22.061979 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7c091-d382-4da5-85de-29b7f621a162 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:22.081986 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7451c3-2ada-436c-a659-0d0072b808d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:22.082763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7451c3-2ada-436c-a659-0d0072b808d2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:22.083047 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7451c3-2ada-436c-a659-0d0072b808d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:22.083323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7451c3-2ada-436c-a659-0d0072b808d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:22.084488 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7451c3-2ada-436c-a659-0d0072b808d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:22.085935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7451c3-2ada-436c-a659-0d0072b808d2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:22.087577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 108/215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:22.093317 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7451c3-2ada-436c-a659-0d0072b808d2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:22.167381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7c091-d382-4da5-85de-29b7f621a162 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:22.180957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:22.198320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7451c3-2ada-436c-a659-0d0072b808d2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:22.206737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5057c-8134-4177-be74-685d62114140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:22.207863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5057c-8134-4177-be74-685d62114140 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:22.208350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5057c-8134-4177-be74-685d62114140 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:22.208828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5057c-8134-4177-be74-685d62114140 None admin] PATH_INFO: `/v3/users/369b62645c344461bcdfa57a1442177e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:22.223311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5057c-8134-4177-be74-685d62114140 None admin] RBAC: Authorizing `identity:get_user(user_id=369b62645c344461bcdfa57a1442177e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:22.225108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5057c-8134-4177-be74-685d62114140 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:22.227091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 108/216] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:22 2022] GET /identity/v3/users/369b62645c344461bcdfa57a1442177e => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:22.232641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5057c-8134-4177-be74-685d62114140 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:22.234985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a526f-9a97-42be-adfa-9cb9002bbe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:22.235667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a526f-9a97-42be-adfa-9cb9002bbe2c None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:22.235941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a526f-9a97-42be-adfa-9cb9002bbe2c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:22.236204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a526f-9a97-42be-adfa-9cb9002bbe2c None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:22.236911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a526f-9a97-42be-adfa-9cb9002bbe2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:22.238359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a526f-9a97-42be-adfa-9cb9002bbe2c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:22.262977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 109/217] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:50:22 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:50:22.272178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:22.346972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5057c-8134-4177-be74-685d62114140 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:22.387349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa876dd-b970-42be-8dd4-2daf64a6343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:22.388019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa876dd-b970-42be-8dd4-2daf64a6343a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:22.388310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa876dd-b970-42be-8dd4-2daf64a6343a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:22.388594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa876dd-b970-42be-8dd4-2daf64a6343a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:22.467232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa876dd-b970-42be-8dd4-2daf64a6343a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:22.468801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa876dd-b970-42be-8dd4-2daf64a6343a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:22.503817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:22.575071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 109/218] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:50:22.580473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a526f-9a97-42be-adfa-9cb9002bbe2c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:22.697173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a526f-9a97-42be-adfa-9cb9002bbe2c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:22.733716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb7db9-66fe-4d22-8923-1df4331c4389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:22.734618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb7db9-66fe-4d22-8923-1df4331c4389 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:22.734907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb7db9-66fe-4d22-8923-1df4331c4389 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:22.735186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb7db9-66fe-4d22-8923-1df4331c4389 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:22.736377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb7db9-66fe-4d22-8923-1df4331c4389 None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:22.737831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb7db9-66fe-4d22-8923-1df4331c4389 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:22.739625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 110/219] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:22 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:22.745859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa876dd-b970-42be-8dd4-2daf64a6343a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:22.873778 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa876dd-b970-42be-8dd4-2daf64a6343a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:22.926757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96ea4f-ebd5-4ac4-92ca-3047356079f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:22.927425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96ea4f-ebd5-4ac4-92ca-3047356079f1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:22.927712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96ea4f-ebd5-4ac4-92ca-3047356079f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:22.927995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96ea4f-ebd5-4ac4-92ca-3047356079f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:22.928841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96ea4f-ebd5-4ac4-92ca-3047356079f1 None admin] RBAC: Adding query filter params (role.id=d3bdf57150744faf9c5c25c317df48fc, scope.project.id=42dcbe790c2a48f2bd52298af1b58573, user.id=369b62645c344461bcdfa57a1442177e) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:22.929161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96ea4f-ebd5-4ac4-92ca-3047356079f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:22.930648 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96ea4f-ebd5-4ac4-92ca-3047356079f1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:22.942769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:22.942916 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 110/220] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:22 2022] GET /identity/v3/role_assignments?user.id=369b62645c344461bcdfa57a1442177e&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:23.018915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fce4e2-b94d-481e-ab0a-e53922f66ee8 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:23.019215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fce4e2-b94d-481e-ab0a-e53922f66ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:23.019402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fce4e2-b94d-481e-ab0a-e53922f66ee8 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:23.021129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 111/221] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:23.026664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16124c62-e883-46fc-87a5-5960a77cf51d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:23.026959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16124c62-e883-46fc-87a5-5960a77cf51d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:23.027245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16124c62-e883-46fc-87a5-5960a77cf51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:23.048281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-16124c62-e883-46fc-87a5-5960a77cf51d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:23.049516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-16124c62-e883-46fc-87a5-5960a77cf51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:23.142944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 111/222] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:23.178365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fce4e2-b94d-481e-ab0a-e53922f66ee8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:23.197277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:23.342011 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08a785-d2a4-4293-bb7c-b50475507b3e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:23.342339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08a785-d2a4-4293-bb7c-b50475507b3e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:23.342606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08a785-d2a4-4293-bb7c-b50475507b3e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:23.344243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 112/223] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:23.356093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fce4e2-b94d-481e-ab0a-e53922f66ee8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:23.363890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7d475-29f9-41c6-8928-87bfe6c9217b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:23.364183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7d475-29f9-41c6-8928-87bfe6c9217b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:23.364472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7d475-29f9-41c6-8928-87bfe6c9217b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:23.397729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563b061-52b2-4c6d-9330-a8345c4f50b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:23.398295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-19c7d475-29f9-41c6-8928-87bfe6c9217b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:23.398577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563b061-52b2-4c6d-9330-a8345c4f50b1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:23.398867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563b061-52b2-4c6d-9330-a8345c4f50b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:23.399149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7563b061-52b2-4c6d-9330-a8345c4f50b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:23.399523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-19c7d475-29f9-41c6-8928-87bfe6c9217b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:23.417645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7563b061-52b2-4c6d-9330-a8345c4f50b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7563b061-52b2-4c6d-9330-a8345c4f50b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:23.425992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:23.426921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:23.427733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:23.429054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 112/224] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:23.437224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563b061-52b2-4c6d-9330-a8345c4f50b1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:23.541036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 113/225] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:23.566519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:23.587560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7d475-29f9-41c6-8928-87bfe6c9217b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:23.620701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7563b061-52b2-4c6d-9330-a8345c4f50b1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:23.658913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cb46e-05e3-4ead-a578-0c048e55c3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:23.660171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cb46e-05e3-4ead-a578-0c048e55c3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:23.660663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cb46e-05e3-4ead-a578-0c048e55c3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:23.661144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5cb46e-05e3-4ead-a578-0c048e55c3b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:23.662086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cb46e-05e3-4ead-a578-0c048e55c3b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:23.662642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cb46e-05e3-4ead-a578-0c048e55c3b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:23.664232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5cb46e-05e3-4ead-a578-0c048e55c3b6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:23.674831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 113/226] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:23.684458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cb46e-05e3-4ead-a578-0c048e55c3b6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:23.734975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:23.804837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7d475-29f9-41c6-8928-87bfe6c9217b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:23.871775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26549ee0-baf4-4885-a57d-648330faa2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:23.872810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26549ee0-baf4-4885-a57d-648330faa2fd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:23.873099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26549ee0-baf4-4885-a57d-648330faa2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:23.873381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26549ee0-baf4-4885-a57d-648330faa2fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:23.885661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5cb46e-05e3-4ead-a578-0c048e55c3b6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:23.896985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26549ee0-baf4-4885-a57d-648330faa2fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-26549ee0-baf4-4885-a57d-648330faa2fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:23.909738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:23.910715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:23.911520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:23.912132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 114/227] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:50:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:23.920892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26549ee0-baf4-4885-a57d-648330faa2fd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:23.959761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:23.984413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13e7c7-d273-49b7-b627-c306a499490d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:23.985648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13e7c7-d273-49b7-b627-c306a499490d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:23.986166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13e7c7-d273-49b7-b627-c306a499490d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:23.986670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc13e7c7-d273-49b7-b627-c306a499490d None admin] PATH_INFO: `/v3/users/a009d22d7fc84a9c95fb35ac1635b02c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:23.991173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13e7c7-d273-49b7-b627-c306a499490d None admin] RBAC: Authorizing `identity:get_user(user_id=a009d22d7fc84a9c95fb35ac1635b02c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:23.992857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc13e7c7-d273-49b7-b627-c306a499490d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:23.994856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 114/228] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:23 2022] GET /identity/v3/users/a009d22d7fc84a9c95fb35ac1635b02c => generated 283 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:24.015456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13e7c7-d273-49b7-b627-c306a499490d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:24.042202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26549ee0-baf4-4885-a57d-648330faa2fd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:24.070783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:24.080673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b59dca-d2f9-4d76-82de-7bec48897404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:24.081384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b59dca-d2f9-4d76-82de-7bec48897404 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:24.082178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b59dca-d2f9-4d76-82de-7bec48897404 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:24.082481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b59dca-d2f9-4d76-82de-7bec48897404 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:24.083406 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b59dca-d2f9-4d76-82de-7bec48897404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:24.083727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b59dca-d2f9-4d76-82de-7bec48897404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:24.085511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b59dca-d2f9-4d76-82de-7bec48897404 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:24.095140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 115/229] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:24.101018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b59dca-d2f9-4d76-82de-7bec48897404 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:24.215462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b59dca-d2f9-4d76-82de-7bec48897404 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:24.250670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e030b-0ae3-4481-97be-d9a6acd8cf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:24.251932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e030b-0ae3-4481-97be-d9a6acd8cf3c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:24.252389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e030b-0ae3-4481-97be-d9a6acd8cf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:24.252833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e030b-0ae3-4481-97be-d9a6acd8cf3c None admin] PATH_INFO: `/v3/users/a009d22d7fc84a9c95fb35ac1635b02c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:24.254184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e030b-0ae3-4481-97be-d9a6acd8cf3c None admin] RBAC: Authorizing `identity:get_user(user_id=a009d22d7fc84a9c95fb35ac1635b02c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:24.256031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e030b-0ae3-4481-97be-d9a6acd8cf3c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:24.257959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 116/230] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:24 2022] GET /identity/v3/users/a009d22d7fc84a9c95fb35ac1635b02c => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:24.265739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e030b-0ae3-4481-97be-d9a6acd8cf3c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:24.297810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc13e7c7-d273-49b7-b627-c306a499490d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:24.343742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:24.377360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857eb6-3f07-456d-8f6c-bec24af2d236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:24.378228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857eb6-3f07-456d-8f6c-bec24af2d236 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:24.378540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857eb6-3f07-456d-8f6c-bec24af2d236 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:24.378825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857eb6-3f07-456d-8f6c-bec24af2d236 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:24.383210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be857eb6-3f07-456d-8f6c-bec24af2d236 None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:24.384680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be857eb6-3f07-456d-8f6c-bec24af2d236 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:24.392436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 115/231] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:24 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:24.398505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857eb6-3f07-456d-8f6c-bec24af2d236 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:24.441446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e030b-0ae3-4481-97be-d9a6acd8cf3c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:24.490799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475d35c-1d90-4ea4-9a79-b760950543ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:24.491598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475d35c-1d90-4ea4-9a79-b760950543ed None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:24.491887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475d35c-1d90-4ea4-9a79-b760950543ed None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:24.492170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475d35c-1d90-4ea4-9a79-b760950543ed None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:24.495409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475d35c-1d90-4ea4-9a79-b760950543ed None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:24.496892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475d35c-1d90-4ea4-9a79-b760950543ed None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:24.501557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 117/232] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:24 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:24.511441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475d35c-1d90-4ea4-9a79-b760950543ed None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:24.551857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:24.616360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857eb6-3f07-456d-8f6c-bec24af2d236 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:24.678864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76828d58-bf2c-432b-9700-8b5088f7188c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:24.680215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76828d58-bf2c-432b-9700-8b5088f7188c None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:24.680711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76828d58-bf2c-432b-9700-8b5088f7188c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:24.681191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76828d58-bf2c-432b-9700-8b5088f7188c None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573/users/a009d22d7fc84a9c95fb35ac1635b02c/roles/fa67cb2f211d433097f606ff029f863e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:24.687327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76828d58-bf2c-432b-9700-8b5088f7188c None admin] RBAC: Authorizing `identity:create_grant(project_id=42dcbe790c2a48f2bd52298af1b58573, user_id=a009d22d7fc84a9c95fb35ac1635b02c, role_id=fa67cb2f211d433097f606ff029f863e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:24.690195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76828d58-bf2c-432b-9700-8b5088f7188c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:24.697316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475d35c-1d90-4ea4-9a79-b760950543ed None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:24.719825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 116/233] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:50:24 2022] PUT /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573/users/a009d22d7fc84a9c95fb35ac1635b02c/roles/fa67cb2f211d433097f606ff029f863e => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:50:24.734712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:24.761220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf329000-83ba-4c5e-891a-a2935dcf2e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:24.763340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf329000-83ba-4c5e-891a-a2935dcf2e87 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:24.764077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf329000-83ba-4c5e-891a-a2935dcf2e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:24.764778 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf329000-83ba-4c5e-891a-a2935dcf2e87 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573/users/a009d22d7fc84a9c95fb35ac1635b02c/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:24.771990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf329000-83ba-4c5e-891a-a2935dcf2e87 None admin] RBAC: Authorizing `identity:create_grant(project_id=42dcbe790c2a48f2bd52298af1b58573, user_id=a009d22d7fc84a9c95fb35ac1635b02c, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:24.774237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf329000-83ba-4c5e-891a-a2935dcf2e87 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:24.817399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 118/234] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:50:24 2022] PUT /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573/users/a009d22d7fc84a9c95fb35ac1635b02c/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:50:24.912467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4737ec-f97f-40c9-a80c-20678fb2dce6 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:24.912802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4737ec-f97f-40c9-a80c-20678fb2dce6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:24.913042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4737ec-f97f-40c9-a80c-20678fb2dce6 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:24.914636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 117/235] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:24.921358 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4dd3b-d3ed-4618-9a8d-6529fb0705c2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:24.921664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4dd3b-d3ed-4618-9a8d-6529fb0705c2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:24.921937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4dd3b-d3ed-4618-9a8d-6529fb0705c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:24.943630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6ed4dd3b-d3ed-4618-9a8d-6529fb0705c2 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:24.944921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6ed4dd3b-d3ed-4618-9a8d-6529fb0705c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:25.038382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 119/236] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:25.063217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4737ec-f97f-40c9-a80c-20678fb2dce6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:25.176735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f6129-9666-4b0d-994b-ee0418117163 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:25.177049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f6129-9666-4b0d-994b-ee0418117163 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:25.177298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f6129-9666-4b0d-994b-ee0418117163 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:25.178867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 120/237] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:25.184948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e253e-0888-4735-8579-7e96f7e7ffdd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:25.185236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e253e-0888-4735-8579-7e96f7e7ffdd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:25.185519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e253e-0888-4735-8579-7e96f7e7ffdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:25.191923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4737ec-f97f-40c9-a80c-20678fb2dce6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:25.212419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-147e253e-0888-4735-8579-7e96f7e7ffdd None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:25.214702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-147e253e-0888-4735-8579-7e96f7e7ffdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:25.242007 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:25.256616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dea0b-fbba-4d0d-b447-372045ad030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:25.257415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dea0b-fbba-4d0d-b447-372045ad030b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:25.257696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dea0b-fbba-4d0d-b447-372045ad030b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:25.257969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dea0b-fbba-4d0d-b447-372045ad030b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:25.277282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0dea0b-fbba-4d0d-b447-372045ad030b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0d0dea0b-fbba-4d0d-b447-372045ad030b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:25.293928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:25.294885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:25.295686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 118/238] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:25.303872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dea0b-fbba-4d0d-b447-372045ad030b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:25.346306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 121/239] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:25.403045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e253e-0888-4735-8579-7e96f7e7ffdd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:25.465506 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:25.487381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dea0b-fbba-4d0d-b447-372045ad030b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:25.548350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ea5fc-7bfe-4a85-a1fd-698249e489eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:25.548957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ea5fc-7bfe-4a85-a1fd-698249e489eb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:25.549890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ea5fc-7bfe-4a85-a1fd-698249e489eb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:25.550210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ea5fc-7bfe-4a85-a1fd-698249e489eb None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:25.551052 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ea5fc-7bfe-4a85-a1fd-698249e489eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:25.551368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ea5fc-7bfe-4a85-a1fd-698249e489eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:25.552794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3ea5fc-7bfe-4a85-a1fd-698249e489eb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:25.565828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 119/240] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:25.573406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ea5fc-7bfe-4a85-a1fd-698249e489eb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:25.602758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e253e-0888-4735-8579-7e96f7e7ffdd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:25.641758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736299bf-100c-4d73-b005-8ecd4ea467ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:25.642976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736299bf-100c-4d73-b005-8ecd4ea467ef None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:25.643468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736299bf-100c-4d73-b005-8ecd4ea467ef None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:25.643948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736299bf-100c-4d73-b005-8ecd4ea467ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:25.648141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736299bf-100c-4d73-b005-8ecd4ea467ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:25.651163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736299bf-100c-4d73-b005-8ecd4ea467ef None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:25.653210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 122/241] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:25.658777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736299bf-100c-4d73-b005-8ecd4ea467ef None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:25.678259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:25.760927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3ea5fc-7bfe-4a85-a1fd-698249e489eb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:25.796569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d901b75-6ccb-4ff8-9b8a-509bfc50a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:25.797441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d901b75-6ccb-4ff8-9b8a-509bfc50a016 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:25.797714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d901b75-6ccb-4ff8-9b8a-509bfc50a016 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:25.797988 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d901b75-6ccb-4ff8-9b8a-509bfc50a016 None admin] PATH_INFO: `/v3/users/e6242c39e4f542eda5268fd472ac9380` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:25.811705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d901b75-6ccb-4ff8-9b8a-509bfc50a016 None admin] RBAC: Authorizing `identity:get_user(user_id=e6242c39e4f542eda5268fd472ac9380)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:25.813168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d901b75-6ccb-4ff8-9b8a-509bfc50a016 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:25.814935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 120/242] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:25 2022] GET /identity/v3/users/e6242c39e4f542eda5268fd472ac9380 => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:25.816167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccc648-cde2-42a6-8889-76804a91aee0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:25.816458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccc648-cde2-42a6-8889-76804a91aee0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:25.816739 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ccc648-cde2-42a6-8889-76804a91aee0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:25.817962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 121/243] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:25.822173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736299bf-100c-4d73-b005-8ecd4ea467ef None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:25.823336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e403be-d4bf-4b58-95a4-f9e2df793f05 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:25.823625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e403be-d4bf-4b58-95a4-f9e2df793f05 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:25.823929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e403be-d4bf-4b58-95a4-f9e2df793f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:25.834202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:25.845683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-16e403be-d4bf-4b58-95a4-f9e2df793f05 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:25.847055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-16e403be-d4bf-4b58-95a4-f9e2df793f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:25.870151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c8df3-8e4a-4fb2-8bf0-6ce9c6feece9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:25.871249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c8df3-8e4a-4fb2-8bf0-6ce9c6feece9 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:25.871780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c8df3-8e4a-4fb2-8bf0-6ce9c6feece9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:25.872257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c8df3-8e4a-4fb2-8bf0-6ce9c6feece9 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:25.873185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c8df3-8e4a-4fb2-8bf0-6ce9c6feece9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:25.874983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0c8df3-8e4a-4fb2-8bf0-6ce9c6feece9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:25.904016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 123/244] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:50:25 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:25.904874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c8df3-8e4a-4fb2-8bf0-6ce9c6feece9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:25.940961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 122/245] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:25.967085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e403be-d4bf-4b58-95a4-f9e2df793f05 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:26.052405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:26.114955 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c8df3-8e4a-4fb2-8bf0-6ce9c6feece9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:26.148454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e403be-d4bf-4b58-95a4-f9e2df793f05 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:26.155131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689233c-d204-4712-b2ad-294f68dcd537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:26.155788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689233c-d204-4712-b2ad-294f68dcd537 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:26.156074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689233c-d204-4712-b2ad-294f68dcd537 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:26.156355 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689233c-d204-4712-b2ad-294f68dcd537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:26.190975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:26.197092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a150a-9ae0-4a98-9854-974b25e2e27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:26.199956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a150a-9ae0-4a98-9854-974b25e2e27d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:26.200479 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a150a-9ae0-4a98-9854-974b25e2e27d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:26.200959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a150a-9ae0-4a98-9854-974b25e2e27d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:26.215796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44a150a-9ae0-4a98-9854-974b25e2e27d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e44a150a-9ae0-4a98-9854-974b25e2e27d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:26.226525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:26.227410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:26.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:26.231552 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 123/246] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:26.237391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a150a-9ae0-4a98-9854-974b25e2e27d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:26.238762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689233c-d204-4712-b2ad-294f68dcd537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:26.240223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689233c-d204-4712-b2ad-294f68dcd537 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:26.370093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:26.372617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 124/247] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:26.377130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44a150a-9ae0-4a98-9854-974b25e2e27d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:26.381960 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689233c-d204-4712-b2ad-294f68dcd537 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:26.421444 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825384e-d96d-4258-8e83-f938390fd594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:26.422075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825384e-d96d-4258-8e83-f938390fd594 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:26.422400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825384e-d96d-4258-8e83-f938390fd594 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:26.422682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825384e-d96d-4258-8e83-f938390fd594 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:26.423435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825384e-d96d-4258-8e83-f938390fd594 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:26.423748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825384e-d96d-4258-8e83-f938390fd594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:26.425217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825384e-d96d-4258-8e83-f938390fd594 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:26.437890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 124/248] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:26.443320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825384e-d96d-4258-8e83-f938390fd594 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:26.482792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:26.521546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689233c-d204-4712-b2ad-294f68dcd537 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:26.563808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c815c66-339d-41fd-b429-7f08e314a4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:26.564678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815c66-339d-41fd-b429-7f08e314a4cf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:26.564965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815c66-339d-41fd-b429-7f08e314a4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:26.565244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c815c66-339d-41fd-b429-7f08e314a4cf None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:26.566470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c815c66-339d-41fd-b429-7f08e314a4cf None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:26.568002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c815c66-339d-41fd-b429-7f08e314a4cf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:26.575144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 125/249] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:26 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:26.576636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c815c66-339d-41fd-b429-7f08e314a4cf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:26.578058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825384e-d96d-4258-8e83-f938390fd594 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:26.622182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb14db9-cc61-4ac6-8386-d421cbc2c5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:26.623035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb14db9-cc61-4ac6-8386-d421cbc2c5eb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:26.623324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb14db9-cc61-4ac6-8386-d421cbc2c5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:26.623602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb14db9-cc61-4ac6-8386-d421cbc2c5eb None admin] PATH_INFO: `/v3/users/369b62645c344461bcdfa57a1442177e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:26.625406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb14db9-cc61-4ac6-8386-d421cbc2c5eb None admin] RBAC: Authorizing `identity:get_user(user_id=369b62645c344461bcdfa57a1442177e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:26.627279 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb14db9-cc61-4ac6-8386-d421cbc2c5eb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:26.630012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 125/250] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:26 2022] GET /identity/v3/users/369b62645c344461bcdfa57a1442177e => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:26.636730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb14db9-cc61-4ac6-8386-d421cbc2c5eb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:26.672376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:26.710120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c815c66-339d-41fd-b429-7f08e314a4cf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:26.743184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:26.749772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27ef45-683f-42ce-bd58-883c0dd74997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:26.750666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27ef45-683f-42ce-bd58-883c0dd74997 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:26.751020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27ef45-683f-42ce-bd58-883c0dd74997 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:26.751304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27ef45-683f-42ce-bd58-883c0dd74997 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:26.752483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27ef45-683f-42ce-bd58-883c0dd74997 None admin] RBAC: Adding query filter params (role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc, scope.project.id=42dcbe790c2a48f2bd52298af1b58573, user.id=e6242c39e4f542eda5268fd472ac9380) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:26.752853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27ef45-683f-42ce-bd58-883c0dd74997 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:26.755608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27ef45-683f-42ce-bd58-883c0dd74997 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:26.762983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 126/251] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:26 2022] GET /identity/v3/role_assignments?user.id=e6242c39e4f542eda5268fd472ac9380&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:26.810533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb14db9-cc61-4ac6-8386-d421cbc2c5eb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:26.848700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91968a01-ccea-43e0-a450-4a9b1fcd8c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:26.849620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91968a01-ccea-43e0-a450-4a9b1fcd8c96 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:26.849911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91968a01-ccea-43e0-a450-4a9b1fcd8c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:26.850191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91968a01-ccea-43e0-a450-4a9b1fcd8c96 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:26.851412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91968a01-ccea-43e0-a450-4a9b1fcd8c96 None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:26.852871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91968a01-ccea-43e0-a450-4a9b1fcd8c96 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:26.855231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 126/252] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:26 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:26.861116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27ef45-683f-42ce-bd58-883c0dd74997 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:27.010839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27ef45-683f-42ce-bd58-883c0dd74997 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:27.063826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3df88-9c6a-473f-ac12-61db5cdb1316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:27.064746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3df88-9c6a-473f-ac12-61db5cdb1316 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:27.065029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3df88-9c6a-473f-ac12-61db5cdb1316 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:27.065310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3df88-9c6a-473f-ac12-61db5cdb1316 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573/users/369b62645c344461bcdfa57a1442177e/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:27.067405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3df88-9c6a-473f-ac12-61db5cdb1316 None admin] RBAC: Authorizing `identity:create_grant(project_id=42dcbe790c2a48f2bd52298af1b58573, user_id=369b62645c344461bcdfa57a1442177e, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:27.068918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3df88-9c6a-473f-ac12-61db5cdb1316 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:27.105579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 127/253] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:50:26 2022] PUT /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573/users/369b62645c344461bcdfa57a1442177e/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:50:27.237256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008c88e-27c5-4104-bda3-8f8ae67aec49 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:27.237627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008c88e-27c5-4104-bda3-8f8ae67aec49 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:27.237819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008c88e-27c5-4104-bda3-8f8ae67aec49 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:27.239240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 127/254] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:27.245115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1956bd-bb06-4f2c-b9f1-0023502f3074 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:27.245444 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1956bd-bb06-4f2c-b9f1-0023502f3074 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:27.245669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1956bd-bb06-4f2c-b9f1-0023502f3074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:27.271100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-be1956bd-bb06-4f2c-b9f1-0023502f3074 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:27.272252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-be1956bd-bb06-4f2c-b9f1-0023502f3074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:27.348973 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:27.369507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 128/255] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:27.405728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008c88e-27c5-4104-bda3-8f8ae67aec49 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:27.522004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:27.533122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008c88e-27c5-4104-bda3-8f8ae67aec49 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:27.585044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95f6c8-0728-4e33-925d-e509bf1639a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:27.586182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95f6c8-0728-4e33-925d-e509bf1639a3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:27.586489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95f6c8-0728-4e33-925d-e509bf1639a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:27.586777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95f6c8-0728-4e33-925d-e509bf1639a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:27.604389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95f6c8-0728-4e33-925d-e509bf1639a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-cb95f6c8-0728-4e33-925d-e509bf1639a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:27.612765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:27.614816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:27.616737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 128/256] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:27.622473 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1956bd-bb06-4f2c-b9f1-0023502f3074 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:27.758494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1956bd-bb06-4f2c-b9f1-0023502f3074 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:27.821856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d4c05-9a6c-4bf5-b7ca-006e00f9d956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:27.823115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d4c05-9a6c-4bf5-b7ca-006e00f9d956 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:27.823403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d4c05-9a6c-4bf5-b7ca-006e00f9d956 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:27.823680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d4c05-9a6c-4bf5-b7ca-006e00f9d956 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:27.824438 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d4c05-9a6c-4bf5-b7ca-006e00f9d956 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:27.824745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d4c05-9a6c-4bf5-b7ca-006e00f9d956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:27.826150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d4c05-9a6c-4bf5-b7ca-006e00f9d956 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:27.838224 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 129/257] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:27.841645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066b4d0-97c5-4ba7-9c6a-656c57f9b8a4 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:27.841944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066b4d0-97c5-4ba7-9c6a-656c57f9b8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:27.842229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066b4d0-97c5-4ba7-9c6a-656c57f9b8a4 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:27.843722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d4c05-9a6c-4bf5-b7ca-006e00f9d956 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:27.844273 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 129/258] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:27.850901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335524-7ce6-4782-ad34-bbbc5a9ac27b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:27.851199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335524-7ce6-4782-ad34-bbbc5a9ac27b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:27.851494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335524-7ce6-4782-ad34-bbbc5a9ac27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:27.891520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c0335524-7ce6-4782-ad34-bbbc5a9ac27b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:27.892761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c0335524-7ce6-4782-ad34-bbbc5a9ac27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:27.931528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:28.050326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d4c05-9a6c-4bf5-b7ca-006e00f9d956 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:28.061153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 130/259] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:28.107862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0335524-7ce6-4782-ad34-bbbc5a9ac27b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:28.117471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101a195-6867-4996-a1d7-8fc65513fdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:28.118659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101a195-6867-4996-a1d7-8fc65513fdea None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:28.119698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101a195-6867-4996-a1d7-8fc65513fdea None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:28.120176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101a195-6867-4996-a1d7-8fc65513fdea None admin] PATH_INFO: `/v3/users/a009d22d7fc84a9c95fb35ac1635b02c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:28.127190 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101a195-6867-4996-a1d7-8fc65513fdea None admin] RBAC: Authorizing `identity:get_user(user_id=a009d22d7fc84a9c95fb35ac1635b02c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:28.128886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101a195-6867-4996-a1d7-8fc65513fdea None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:28.130783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 130/260] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:27 2022] GET /identity/v3/users/a009d22d7fc84a9c95fb35ac1635b02c => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:28.137493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101a195-6867-4996-a1d7-8fc65513fdea None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:28.311146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101a195-6867-4996-a1d7-8fc65513fdea None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:28.319109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:28.333024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:28.352586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0335524-7ce6-4782-ad34-bbbc5a9ac27b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:28.368231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab5b7f-b393-4b18-a67d-e42c25fd95cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:28.369810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab5b7f-b393-4b18-a67d-e42c25fd95cb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:28.370294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab5b7f-b393-4b18-a67d-e42c25fd95cb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:28.370803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab5b7f-b393-4b18-a67d-e42c25fd95cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:28.453661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779c084-e7be-48e9-b3ae-ed82c39f84c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:28.454502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779c084-e7be-48e9-b3ae-ed82c39f84c2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:28.454795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779c084-e7be-48e9-b3ae-ed82c39f84c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:28.455075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779c084-e7be-48e9-b3ae-ed82c39f84c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:28.474170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab5b7f-b393-4b18-a67d-e42c25fd95cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:28.483806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab5b7f-b393-4b18-a67d-e42c25fd95cb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:28.498469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0779c084-e7be-48e9-b3ae-ed82c39f84c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0779c084-e7be-48e9-b3ae-ed82c39f84c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:28.503197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:28.504082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:28.504878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 131/261] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:50:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:28.519434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779c084-e7be-48e9-b3ae-ed82c39f84c2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:28.615441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 131/262] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:50:28.623507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab5b7f-b393-4b18-a67d-e42c25fd95cb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:28.658807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:28.756092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779c084-e7be-48e9-b3ae-ed82c39f84c2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:28.823845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab5b7f-b393-4b18-a67d-e42c25fd95cb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:28.839195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f52c1-e556-4c6d-ab3c-ec3a409a7ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:28.839890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f52c1-e556-4c6d-ab3c-ec3a409a7ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:28.840180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f52c1-e556-4c6d-ab3c-ec3a409a7ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:28.840462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f52c1-e556-4c6d-ab3c-ec3a409a7ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:28.841214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f52c1-e556-4c6d-ab3c-ec3a409a7ed4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:28.841527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f52c1-e556-4c6d-ab3c-ec3a409a7ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:28.842956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f52c1-e556-4c6d-ab3c-ec3a409a7ed4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:28.852118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 132/263] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:28.858104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f52c1-e556-4c6d-ab3c-ec3a409a7ed4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:28.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:28.910288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae4625-c61d-4d00-b003-5ce6c945e92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:28.911247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae4625-c61d-4d00-b003-5ce6c945e92d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:28.911538 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae4625-c61d-4d00-b003-5ce6c945e92d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:28.911893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae4625-c61d-4d00-b003-5ce6c945e92d None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:28.913105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae4625-c61d-4d00-b003-5ce6c945e92d None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:28.914687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae4625-c61d-4d00-b003-5ce6c945e92d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:28.916466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 132/264] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:28 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:28.927148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae4625-c61d-4d00-b003-5ce6c945e92d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:29.011552 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f52c1-e556-4c6d-ab3c-ec3a409a7ed4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:29.064098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2270bf-b09d-4fd7-91cc-65f684a6be06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:29.064992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2270bf-b09d-4fd7-91cc-65f684a6be06 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:29.065276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2270bf-b09d-4fd7-91cc-65f684a6be06 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:29.065554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2270bf-b09d-4fd7-91cc-65f684a6be06 None admin] PATH_INFO: `/v3/users/a009d22d7fc84a9c95fb35ac1635b02c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:29.071214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2270bf-b09d-4fd7-91cc-65f684a6be06 None admin] RBAC: Authorizing `identity:get_user(user_id=a009d22d7fc84a9c95fb35ac1635b02c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:29.072760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2270bf-b09d-4fd7-91cc-65f684a6be06 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:29.074781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 133/265] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:28 2022] GET /identity/v3/users/a009d22d7fc84a9c95fb35ac1635b02c => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:29.080414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2270bf-b09d-4fd7-91cc-65f684a6be06 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:29.143889 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:29.191515 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:29.220491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae4625-c61d-4d00-b003-5ce6c945e92d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:29.234033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2270bf-b09d-4fd7-91cc-65f684a6be06 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:29.300554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124d80e-0a3a-4d3f-9dc3-403128037b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:29.301222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124d80e-0a3a-4d3f-9dc3-403128037b9e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:29.301511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124d80e-0a3a-4d3f-9dc3-403128037b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:29.301793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124d80e-0a3a-4d3f-9dc3-403128037b9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:29.332099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5164154-e80d-4786-8ffc-9239e1984261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:29.332770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5164154-e80d-4786-8ffc-9239e1984261 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:29.333062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5164154-e80d-4786-8ffc-9239e1984261 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:29.333341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5164154-e80d-4786-8ffc-9239e1984261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:29.334187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5164154-e80d-4786-8ffc-9239e1984261 None admin] RBAC: Adding query filter params (role.id=fa67cb2f211d433097f606ff029f863e, scope.project.id=42dcbe790c2a48f2bd52298af1b58573, user.id=a009d22d7fc84a9c95fb35ac1635b02c) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:29.334528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5164154-e80d-4786-8ffc-9239e1984261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:29.336000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5164154-e80d-4786-8ffc-9239e1984261 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:29.356828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 133/266] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:28 2022] GET /identity/v3/role_assignments?user.id=a009d22d7fc84a9c95fb35ac1635b02c&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=fa67cb2f211d433097f606ff029f863e => generated 608 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:29.414049 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124d80e-0a3a-4d3f-9dc3-403128037b9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:29.415485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124d80e-0a3a-4d3f-9dc3-403128037b9e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:29.469861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74935-25b6-4c97-b8f2-b53372f46f58 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:29.470652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74935-25b6-4c97-b8f2-b53372f46f58 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:29.471141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e74935-25b6-4c97-b8f2-b53372f46f58 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:29.473105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 134/267] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:29.479323 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3663931-6393-4e0d-b626-0c5102bc62bf None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:29.480050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3663931-6393-4e0d-b626-0c5102bc62bf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:29.480524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3663931-6393-4e0d-b626-0c5102bc62bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:29.505805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e3663931-6393-4e0d-b626-0c5102bc62bf None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:29.512777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e3663931-6393-4e0d-b626-0c5102bc62bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:29.543024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:29.596153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 134/268] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:50:29.612762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 135/269] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:29.614757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124d80e-0a3a-4d3f-9dc3-403128037b9e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:29.641346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3663931-6393-4e0d-b626-0c5102bc62bf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:29.791139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:29.822790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124d80e-0a3a-4d3f-9dc3-403128037b9e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:29.845001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3663931-6393-4e0d-b626-0c5102bc62bf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:29.883786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978064fc-0989-41b9-9730-606ad08705ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:29.885849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978064fc-0989-41b9-9730-606ad08705ef None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:29.886841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978064fc-0989-41b9-9730-606ad08705ef None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:29.888024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978064fc-0989-41b9-9730-606ad08705ef None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:29.890086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978064fc-0989-41b9-9730-606ad08705ef None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:29.892254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978064fc-0989-41b9-9730-606ad08705ef None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:29.893049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8731f6c-246e-47ed-83d9-02990c23980b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:29.894062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8731f6c-246e-47ed-83d9-02990c23980b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:29.894542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8731f6c-246e-47ed-83d9-02990c23980b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:29.894993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8731f6c-246e-47ed-83d9-02990c23980b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:29.898552 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 135/270] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:29 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:29.906744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978064fc-0989-41b9-9730-606ad08705ef None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:29.917286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8731f6c-246e-47ed-83d9-02990c23980b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f8731f6c-246e-47ed-83d9-02990c23980b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:29.926803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:29.927722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:29.928525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:29.930904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 136/271] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:29.938479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8731f6c-246e-47ed-83d9-02990c23980b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:30.006863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:30.058239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978064fc-0989-41b9-9730-606ad08705ef None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:30.094291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c673cc41-b3c0-426f-b70d-022908a0859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:30.095221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673cc41-b3c0-426f-b70d-022908a0859d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.095667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8731f6c-246e-47ed-83d9-02990c23980b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:30.095809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673cc41-b3c0-426f-b70d-022908a0859d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.096218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c673cc41-b3c0-426f-b70d-022908a0859d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.097191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673cc41-b3c0-426f-b70d-022908a0859d None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=42dcbe790c2a48f2bd52298af1b58573, user.id=a009d22d7fc84a9c95fb35ac1635b02c) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:30.097654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673cc41-b3c0-426f-b70d-022908a0859d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:30.099286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c673cc41-b3c0-426f-b70d-022908a0859d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:30.105844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 136/272] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:29 2022] GET /identity/v3/role_assignments?user.id=a009d22d7fc84a9c95fb35ac1635b02c&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:30.107281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ef21-efd5-4b54-aa7f-2580ff3ac716 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.107717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ef21-efd5-4b54-aa7f-2580ff3ac716 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.108161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ef21-efd5-4b54-aa7f-2580ff3ac716 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.109845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 137/273] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:30.115622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282f9ff-abc8-44f1-8ad4-9bff55380d30 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.116127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282f9ff-abc8-44f1-8ad4-9bff55380d30 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.116556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282f9ff-abc8-44f1-8ad4-9bff55380d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.143343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2282f9ff-abc8-44f1-8ad4-9bff55380d30 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:30.145145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2282f9ff-abc8-44f1-8ad4-9bff55380d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:30.173019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022b239-3e14-4119-a3ac-1dde5ec3abb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:30.174293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022b239-3e14-4119-a3ac-1dde5ec3abb6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.174820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022b239-3e14-4119-a3ac-1dde5ec3abb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.175270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022b239-3e14-4119-a3ac-1dde5ec3abb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.176300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022b239-3e14-4119-a3ac-1dde5ec3abb6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:30.176870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022b239-3e14-4119-a3ac-1dde5ec3abb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:30.178494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022b239-3e14-4119-a3ac-1dde5ec3abb6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:30.189084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:30.203475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 137/274] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:30.205464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf5eca-df7b-4f67-949b-aa4be5922945 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.205761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf5eca-df7b-4f67-949b-aa4be5922945 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.206036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf5eca-df7b-4f67-949b-aa4be5922945 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.207874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 138/275] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:30.210583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf5eca-df7b-4f67-949b-aa4be5922945 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:30.247780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 138/276] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:30.249503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bec3fc-1413-4521-b49c-31b71954a5ba None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.249801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bec3fc-1413-4521-b49c-31b71954a5ba None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.250180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bec3fc-1413-4521-b49c-31b71954a5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.276939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-69bec3fc-1413-4521-b49c-31b71954a5ba None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:30.279423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-69bec3fc-1413-4521-b49c-31b71954a5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:30.340390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf5eca-df7b-4f67-949b-aa4be5922945 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:30.349757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:30.375774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 139/277] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:30.376587 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bec3fc-1413-4521-b49c-31b71954a5ba None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:30.381958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cceeab-1c7a-43b8-baa5-1fda4aac1fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:30.382863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cceeab-1c7a-43b8-baa5-1fda4aac1fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.383146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cceeab-1c7a-43b8-baa5-1fda4aac1fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.383447 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cceeab-1c7a-43b8-baa5-1fda4aac1fe8 None admin] PATH_INFO: `/v3/users/4f7392485f40434abee66fbdae4c48e4` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.400231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cceeab-1c7a-43b8-baa5-1fda4aac1fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=4f7392485f40434abee66fbdae4c48e4)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:30.400231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cceeab-1c7a-43b8-baa5-1fda4aac1fe8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:30.401881 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 139/278] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:30 2022] GET /identity/v3/users/4f7392485f40434abee66fbdae4c48e4 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:30.403148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277094fc-a6a6-4dbd-bb1e-48b0ca746b8c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.403435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277094fc-a6a6-4dbd-bb1e-48b0ca746b8c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.403706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277094fc-a6a6-4dbd-bb1e-48b0ca746b8c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.404938 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 140/279] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:30.405618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277094fc-a6a6-4dbd-bb1e-48b0ca746b8c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:30.473560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:30.496354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bec3fc-1413-4521-b49c-31b71954a5ba None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:30.517816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277094fc-a6a6-4dbd-bb1e-48b0ca746b8c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:30.529568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8ca30-ee01-4d9e-9cd9-393fc10cca84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:30.530381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8ca30-ee01-4d9e-9cd9-393fc10cca84 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.530698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8ca30-ee01-4d9e-9cd9-393fc10cca84 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.531289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f8ca30-ee01-4d9e-9cd9-393fc10cca84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.532164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8ca30-ee01-4d9e-9cd9-393fc10cca84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:30.533626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f8ca30-ee01-4d9e-9cd9-393fc10cca84 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:30.535332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 140/280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:30.536106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8ca30-ee01-4d9e-9cd9-393fc10cca84 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:30.571695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66d5f4-9f44-4a10-9f38-91a34912f5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:30.572486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66d5f4-9f44-4a10-9f38-91a34912f5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.572762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66d5f4-9f44-4a10-9f38-91a34912f5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.573032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66d5f4-9f44-4a10-9f38-91a34912f5e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.574816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:30.584957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed66d5f4-9f44-4a10-9f38-91a34912f5e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ed66d5f4-9f44-4a10-9f38-91a34912f5e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:30.592831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:30.593713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:30.594556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 141/281] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:30.595738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add50894-bb17-4fa2-8c2f-6a0db45b1732 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.595738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add50894-bb17-4fa2-8c2f-6a0db45b1732 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.595901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add50894-bb17-4fa2-8c2f-6a0db45b1732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.612281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-add50894-bb17-4fa2-8c2f-6a0db45b1732 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:30.613498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-add50894-bb17-4fa2-8c2f-6a0db45b1732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:30.640373 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f8ca30-ee01-4d9e-9cd9-393fc10cca84 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:30.673389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dc2aa-1acb-4883-9cd1-8ed467d724d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:30.674039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dc2aa-1acb-4883-9cd1-8ed467d724d3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.674326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dc2aa-1acb-4883-9cd1-8ed467d724d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.674636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dc2aa-1acb-4883-9cd1-8ed467d724d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.675468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442dc2aa-1acb-4883-9cd1-8ed467d724d3 None admin] RBAC: Adding query filter params (role.id=d3bdf57150744faf9c5c25c317df48fc, user.id=4f7392485f40434abee66fbdae4c48e4, scope.system=all) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:30.675785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442dc2aa-1acb-4883-9cd1-8ed467d724d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:30.677261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442dc2aa-1acb-4883-9cd1-8ed467d724d3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:30.678188 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 142/282] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:30.679019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add50894-bb17-4fa2-8c2f-6a0db45b1732 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:30.689337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 141/283] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 26 20:50:30 2022] GET /identity/v3/role_assignments?user.id=4f7392485f40434abee66fbdae4c48e4&scope.system=all&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:30.690347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dc2aa-1acb-4883-9cd1-8ed467d724d3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:30.739099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:30.803150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:30.806127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dc2aa-1acb-4883-9cd1-8ed467d724d3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:30.808777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add50894-bb17-4fa2-8c2f-6a0db45b1732 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:30.844519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0975eb-be19-4f1f-9e37-cb60baf6daa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:30.845124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0975eb-be19-4f1f-9e37-cb60baf6daa1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.845406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0975eb-be19-4f1f-9e37-cb60baf6daa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.845681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0975eb-be19-4f1f-9e37-cb60baf6daa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.846573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0975eb-be19-4f1f-9e37-cb60baf6daa1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:30.846884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0975eb-be19-4f1f-9e37-cb60baf6daa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:30.848302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0975eb-be19-4f1f-9e37-cb60baf6daa1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:30.856001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34107fdf-bd97-4e36-b6d5-9adc948b2281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:30.856678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34107fdf-bd97-4e36-b6d5-9adc948b2281 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:30.856963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34107fdf-bd97-4e36-b6d5-9adc948b2281 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:30.857238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34107fdf-bd97-4e36-b6d5-9adc948b2281 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:30.857949 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34107fdf-bd97-4e36-b6d5-9adc948b2281 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:30.858565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 142/284] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:30.859349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0975eb-be19-4f1f-9e37-cb60baf6daa1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:30.864766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34107fdf-bd97-4e36-b6d5-9adc948b2281 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:30.886580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 143/285] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:50:30 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:30.887605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34107fdf-bd97-4e36-b6d5-9adc948b2281 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:30.991415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0975eb-be19-4f1f-9e37-cb60baf6daa1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:31.030292 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34107fdf-bd97-4e36-b6d5-9adc948b2281 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:31.032249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:31.036573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64326e13-6de0-41d8-a2a5-c71803f2c557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:31.037704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64326e13-6de0-41d8-a2a5-c71803f2c557 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:31.038153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64326e13-6de0-41d8-a2a5-c71803f2c557 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:31.038699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64326e13-6de0-41d8-a2a5-c71803f2c557 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:31.058205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64326e13-6de0-41d8-a2a5-c71803f2c557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-64326e13-6de0-41d8-a2a5-c71803f2c557 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:31.069959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:31.071151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:31.072193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:31.074501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 143/286] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:31.079186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1568c3-1ab7-4e6d-a681-fcad33756d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:31.079186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1568c3-1ab7-4e6d-a681-fcad33756d0d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:31.079186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1568c3-1ab7-4e6d-a681-fcad33756d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:31.079186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1568c3-1ab7-4e6d-a681-fcad33756d0d None admin] PATH_INFO: `/v3/users/369b62645c344461bcdfa57a1442177e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:31.079753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1568c3-1ab7-4e6d-a681-fcad33756d0d None admin] RBAC: Authorizing `identity:get_user(user_id=369b62645c344461bcdfa57a1442177e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:31.080299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64326e13-6de0-41d8-a2a5-c71803f2c557 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:31.081250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1568c3-1ab7-4e6d-a681-fcad33756d0d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:31.083068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 144/287] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:30 2022] GET /identity/v3/users/369b62645c344461bcdfa57a1442177e => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:31.091452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1568c3-1ab7-4e6d-a681-fcad33756d0d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:31.113718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:31.216850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64326e13-6de0-41d8-a2a5-c71803f2c557 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:31.230844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1568c3-1ab7-4e6d-a681-fcad33756d0d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:31.263657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cf079-e354-4e80-956c-6631893b78a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:31.264353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cf079-e354-4e80-956c-6631893b78a2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:31.264626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cf079-e354-4e80-956c-6631893b78a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:31.264907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cf079-e354-4e80-956c-6631893b78a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:31.265661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cf079-e354-4e80-956c-6631893b78a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:31.265972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cf079-e354-4e80-956c-6631893b78a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:31.267407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8cf079-e354-4e80-956c-6631893b78a2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:31.279197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 144/288] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:31.279359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc4c51-25d7-4082-b905-4fa1bac5cdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:31.279612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc4c51-25d7-4082-b905-4fa1bac5cdbd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:31.279772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc4c51-25d7-4082-b905-4fa1bac5cdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:31.280529 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc4c51-25d7-4082-b905-4fa1bac5cdbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:31.282724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cf079-e354-4e80-956c-6631893b78a2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:31.346334 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:31.356496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc4c51-25d7-4082-b905-4fa1bac5cdbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:31.358021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc4c51-25d7-4082-b905-4fa1bac5cdbd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:31.374088 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:31.391829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cf079-e354-4e80-956c-6631893b78a2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:31.429077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6e4c9-c652-4d45-bb2e-d4b9a7821ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:31.429965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e4c9-c652-4d45-bb2e-d4b9a7821ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:31.430248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e4c9-c652-4d45-bb2e-d4b9a7821ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:31.430558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e4c9-c652-4d45-bb2e-d4b9a7821ed3 None admin] PATH_INFO: `/v3/users/e6242c39e4f542eda5268fd472ac9380` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:31.431814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6e4c9-c652-4d45-bb2e-d4b9a7821ed3 None admin] RBAC: Authorizing `identity:get_user(user_id=e6242c39e4f542eda5268fd472ac9380)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:31.433278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6e4c9-c652-4d45-bb2e-d4b9a7821ed3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:31.436186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 145/289] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:31 2022] GET /identity/v3/users/e6242c39e4f542eda5268fd472ac9380 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:31.441908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6e4c9-c652-4d45-bb2e-d4b9a7821ed3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:31.463759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 145/290] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:31.469545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc4c51-25d7-4082-b905-4fa1bac5cdbd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:31.544380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:31.576241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc4c51-25d7-4082-b905-4fa1bac5cdbd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:31.581559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6e4c9-c652-4d45-bb2e-d4b9a7821ed3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:31.622411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6deef6-c33c-48cd-873b-1225b2fb0d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:31.623565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6deef6-c33c-48cd-873b-1225b2fb0d81 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:31.623946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db85d9-1c37-462e-90e1-c236d4b7d163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:31.624208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6deef6-c33c-48cd-873b-1225b2fb0d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:31.624624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6deef6-c33c-48cd-873b-1225b2fb0d81 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:31.625099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db85d9-1c37-462e-90e1-c236d4b7d163 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:31.625478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db85d9-1c37-462e-90e1-c236d4b7d163 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:31.625857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db85d9-1c37-462e-90e1-c236d4b7d163 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:31.626392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6deef6-c33c-48cd-873b-1225b2fb0d81 None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:31.627248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db85d9-1c37-462e-90e1-c236d4b7d163 None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:31.628435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6deef6-c33c-48cd-873b-1225b2fb0d81 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:31.628848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db85d9-1c37-462e-90e1-c236d4b7d163 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:31.630564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 146/291] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:31 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:31.634709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 146/292] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:31 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:31.636698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6deef6-c33c-48cd-873b-1225b2fb0d81 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:31.643407 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db85d9-1c37-462e-90e1-c236d4b7d163 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:31.689942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:31.795580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db85d9-1c37-462e-90e1-c236d4b7d163 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:31.866433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf544e-d007-4ab2-9e19-cbbc77bed734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:31.867141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf544e-d007-4ab2-9e19-cbbc77bed734 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:31.867432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf544e-d007-4ab2-9e19-cbbc77bed734 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:31.867717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf544e-d007-4ab2-9e19-cbbc77bed734 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:31.868922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf544e-d007-4ab2-9e19-cbbc77bed734 None admin] RBAC: Adding query filter params (role.id=d3bdf57150744faf9c5c25c317df48fc, scope.project.id=42dcbe790c2a48f2bd52298af1b58573, user.id=369b62645c344461bcdfa57a1442177e) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:31.869786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf544e-d007-4ab2-9e19-cbbc77bed734 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:31.871340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf544e-d007-4ab2-9e19-cbbc77bed734 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:31.884166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 147/293] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:31 2022] GET /identity/v3/role_assignments?user.id=369b62645c344461bcdfa57a1442177e&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:31.886736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6deef6-c33c-48cd-873b-1225b2fb0d81 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:31.931284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019464d-ce2c-46bd-ac75-83b640687b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:31.932907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019464d-ce2c-46bd-ac75-83b640687b8f None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:31.933251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019464d-ce2c-46bd-ac75-83b640687b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:31.933539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019464d-ce2c-46bd-ac75-83b640687b8f None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573/users/e6242c39e4f542eda5268fd472ac9380/roles/23aa53c14bd94d60b9e3ffdcfbecd8dc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:31.943846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019464d-ce2c-46bd-ac75-83b640687b8f None admin] RBAC: Authorizing `identity:create_grant(project_id=42dcbe790c2a48f2bd52298af1b58573, user_id=e6242c39e4f542eda5268fd472ac9380, role_id=23aa53c14bd94d60b9e3ffdcfbecd8dc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:31.945358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019464d-ce2c-46bd-ac75-83b640687b8f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:31.971769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 147/294] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:50:31 2022] PUT /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573/users/e6242c39e4f542eda5268fd472ac9380/roles/23aa53c14bd94d60b9e3ffdcfbecd8dc => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:50:33.031488 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe5e4d-e1f6-4270-95d5-56093f235447 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.032034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe5e4d-e1f6-4270-95d5-56093f235447 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.032034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe5e4d-e1f6-4270-95d5-56093f235447 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.033643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 148/295] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:33.038882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e012d7-3961-48f9-aa7a-50cc9f78dd1b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.039296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e012d7-3961-48f9-aa7a-50cc9f78dd1b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.039466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e012d7-3961-48f9-aa7a-50cc9f78dd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.057191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-37e012d7-3961-48f9-aa7a-50cc9f78dd1b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:33.058373 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-37e012d7-3961-48f9-aa7a-50cc9f78dd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:33.127376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 148/296] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:33.150317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe5e4d-e1f6-4270-95d5-56093f235447 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:33.154691 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:33.267999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe5e4d-e1f6-4270-95d5-56093f235447 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:33.304445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6529e1e-7a2b-4965-a96e-0c4ed3401467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:33.305240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6529e1e-7a2b-4965-a96e-0c4ed3401467 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.305524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6529e1e-7a2b-4965-a96e-0c4ed3401467 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.305801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6529e1e-7a2b-4965-a96e-0c4ed3401467 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.317026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6529e1e-7a2b-4965-a96e-0c4ed3401467 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a6529e1e-7a2b-4965-a96e-0c4ed3401467 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:33.325341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:33.326251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:33.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 149/297] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:33.332439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e012d7-3961-48f9-aa7a-50cc9f78dd1b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:33.344404 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:33.441811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e012d7-3961-48f9-aa7a-50cc9f78dd1b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:33.452475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f4c3-38e8-42d1-9a5f-c51c61ea6d59 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.452846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f4c3-38e8-42d1-9a5f-c51c61ea6d59 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.453196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f4c3-38e8-42d1-9a5f-c51c61ea6d59 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.454726 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 150/298] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:33.459517 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04224-80eb-4ba0-a901-ece8201bf525 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.459802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04224-80eb-4ba0-a901-ece8201bf525 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.460079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b04224-80eb-4ba0-a901-ece8201bf525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.477513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-42b04224-80eb-4ba0-a901-ece8201bf525 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:33.479267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-42b04224-80eb-4ba0-a901-ece8201bf525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:33.480244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6c85e-5692-4d37-90b2-bae069ac399b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:33.481136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6c85e-5692-4d37-90b2-bae069ac399b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.481628 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6c85e-5692-4d37-90b2-bae069ac399b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.482116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6c85e-5692-4d37-90b2-bae069ac399b None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.483221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6c85e-5692-4d37-90b2-bae069ac399b None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:33.483729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6c85e-5692-4d37-90b2-bae069ac399b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:33.485335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6c85e-5692-4d37-90b2-bae069ac399b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:33.494803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 149/299] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:33.503855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6c85e-5692-4d37-90b2-bae069ac399b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:33.543615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:33.559610 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 151/300] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:33.582578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b04224-80eb-4ba0-a901-ece8201bf525 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:33.599469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:33.633873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6c85e-5692-4d37-90b2-bae069ac399b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:33.669685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174cff31-c6e3-49f2-81e3-4c431a09f516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:33.670900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cff31-c6e3-49f2-81e3-4c431a09f516 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.671328 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cff31-c6e3-49f2-81e3-4c431a09f516 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.671746 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174cff31-c6e3-49f2-81e3-4c431a09f516 None admin] PATH_INFO: `/v3/users/4f7392485f40434abee66fbdae4c48e4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.673103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174cff31-c6e3-49f2-81e3-4c431a09f516 None admin] RBAC: Authorizing `identity:get_user(user_id=4f7392485f40434abee66fbdae4c48e4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:33.674743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174cff31-c6e3-49f2-81e3-4c431a09f516 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:33.681182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 150/301] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:33 2022] GET /identity/v3/users/4f7392485f40434abee66fbdae4c48e4 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:33.683202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174cff31-c6e3-49f2-81e3-4c431a09f516 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:33.687632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b04224-80eb-4ba0-a901-ece8201bf525 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:33.722912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c037d-982a-4e89-9ce4-d4f5dc0ad031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:33.723655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c037d-982a-4e89-9ce4-d4f5dc0ad031 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.723935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c037d-982a-4e89-9ce4-d4f5dc0ad031 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.724210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c037d-982a-4e89-9ce4-d4f5dc0ad031 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.735727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c037d-982a-4e89-9ce4-d4f5dc0ad031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3c3c037d-982a-4e89-9ce4-d4f5dc0ad031 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:33.743661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:33.745339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:33.746459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:33.750403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 152/302] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:33.751258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c037d-982a-4e89-9ce4-d4f5dc0ad031 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:33.795902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174cff31-c6e3-49f2-81e3-4c431a09f516 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:33.808975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:33.835124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b545f3a-53fd-475e-b318-2546852bef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:33.835996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b545f3a-53fd-475e-b318-2546852bef75 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.836277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b545f3a-53fd-475e-b318-2546852bef75 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.836556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b545f3a-53fd-475e-b318-2546852bef75 None admin] PATH_INFO: `/v3/system/users/4f7392485f40434abee66fbdae4c48e4/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.838585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b545f3a-53fd-475e-b318-2546852bef75 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4f7392485f40434abee66fbdae4c48e4, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:33.839969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b545f3a-53fd-475e-b318-2546852bef75 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:33.849861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 151/303] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 26 20:50:33 2022] PUT /identity/v3/system/users/4f7392485f40434abee66fbdae4c48e4/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:50:33.851800 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29252ef-dbdf-43da-89b0-7fedbe6cfe12 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.852094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29252ef-dbdf-43da-89b0-7fedbe6cfe12 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.852373 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29252ef-dbdf-43da-89b0-7fedbe6cfe12 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.854516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 152/304] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:33.860176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85693311-0ec0-47e2-bd91-0c9ad0ecfe17 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.860479 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85693311-0ec0-47e2-bd91-0c9ad0ecfe17 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.860810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85693311-0ec0-47e2-bd91-0c9ad0ecfe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.867448 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:33.874942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c037d-982a-4e89-9ce4-d4f5dc0ad031 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:33.880597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-85693311-0ec0-47e2-bd91-0c9ad0ecfe17 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:33.881821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-85693311-0ec0-47e2-bd91-0c9ad0ecfe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:33.923567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c2686-91e6-4c89-a8fb-f69c7406d0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:33.924480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c2686-91e6-4c89-a8fb-f69c7406d0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:33.925699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c2686-91e6-4c89-a8fb-f69c7406d0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:33.926002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c2686-91e6-4c89-a8fb-f69c7406d0e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:33.926837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c2686-91e6-4c89-a8fb-f69c7406d0e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:33.927152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c2686-91e6-4c89-a8fb-f69c7406d0e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:33.928689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c2686-91e6-4c89-a8fb-f69c7406d0e4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:33.946641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 153/305] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:33.948563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c2686-91e6-4c89-a8fb-f69c7406d0e4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:33.964562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 153/306] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:33.988406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85693311-0ec0-47e2-bd91-0c9ad0ecfe17 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:34.083962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c2686-91e6-4c89-a8fb-f69c7406d0e4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:34.102746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:34.116565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85693311-0ec0-47e2-bd91-0c9ad0ecfe17 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:34.127522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09caa33-7185-46f0-b2c3-e7b220a4994d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:34.128513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09caa33-7185-46f0-b2c3-e7b220a4994d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:34.128798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09caa33-7185-46f0-b2c3-e7b220a4994d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:34.129084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09caa33-7185-46f0-b2c3-e7b220a4994d None admin] PATH_INFO: `/v3/users/811240db838441d4ba462bd6bfae1964` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:34.143996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09caa33-7185-46f0-b2c3-e7b220a4994d None admin] RBAC: Authorizing `identity:get_user(user_id=811240db838441d4ba462bd6bfae1964)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:34.145705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09caa33-7185-46f0-b2c3-e7b220a4994d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:34.148007 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 154/307] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:33 2022] GET /identity/v3/users/811240db838441d4ba462bd6bfae1964 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:34.153639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168ad1c-95d7-467f-bea6-0833e1c86718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:34.154200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09caa33-7185-46f0-b2c3-e7b220a4994d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:34.154472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168ad1c-95d7-467f-bea6-0833e1c86718 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:34.154836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168ad1c-95d7-467f-bea6-0833e1c86718 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:34.155117 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9168ad1c-95d7-467f-bea6-0833e1c86718 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:34.156334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168ad1c-95d7-467f-bea6-0833e1c86718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:34.158832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9168ad1c-95d7-467f-bea6-0833e1c86718 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:34.160641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 154/308] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:34.168053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168ad1c-95d7-467f-bea6-0833e1c86718 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:34.172825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:34.280255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09caa33-7185-46f0-b2c3-e7b220a4994d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:34.288202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9168ad1c-95d7-467f-bea6-0833e1c86718 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:34.323082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d7530-a807-4d56-a49c-771a518ce7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:34.323742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d7530-a807-4d56-a49c-771a518ce7db None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:34.324026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d7530-a807-4d56-a49c-771a518ce7db None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:34.324301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d7530-a807-4d56-a49c-771a518ce7db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:34.324532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18926852-336f-4d96-8972-8d4e3c029348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:34.325135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d7530-a807-4d56-a49c-771a518ce7db None admin] RBAC: Adding query filter params (role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc, user.id=811240db838441d4ba462bd6bfae1964, scope.system=all) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:34.325323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18926852-336f-4d96-8972-8d4e3c029348 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:34.325476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d7530-a807-4d56-a49c-771a518ce7db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:34.325619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18926852-336f-4d96-8972-8d4e3c029348 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:34.325892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18926852-336f-4d96-8972-8d4e3c029348 None admin] PATH_INFO: `/v3/groups` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:34.326623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18926852-336f-4d96-8972-8d4e3c029348 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:34.326941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d7530-a807-4d56-a49c-771a518ce7db None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:34.328094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18926852-336f-4d96-8972-8d4e3c029348 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:34.332920 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 155/309] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 26 20:50:34 2022] GET /identity/v3/role_assignments?user.id=811240db838441d4ba462bd6bfae1964&scope.system=all&role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:34.351456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 155/310] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 26 20:50:34 2022] POST /identity/v3/groups => generated 229 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:34.365928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948288af-9de5-4c07-9c34-616f33bd52f6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:34.366312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948288af-9de5-4c07-9c34-616f33bd52f6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:34.366497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948288af-9de5-4c07-9c34-616f33bd52f6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:34.368334 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 156/311] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:34.374466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57412fb5-a806-45fb-ad1d-caa16dd291b8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:34.374784 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57412fb5-a806-45fb-ad1d-caa16dd291b8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:34.375020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57412fb5-a806-45fb-ad1d-caa16dd291b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:34.394796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-57412fb5-a806-45fb-ad1d-caa16dd291b8 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:34.396038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-57412fb5-a806-45fb-ad1d-caa16dd291b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:34.472835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:34.488018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 156/312] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:34.512257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948288af-9de5-4c07-9c34-616f33bd52f6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:34.554190 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:34.629377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948288af-9de5-4c07-9c34-616f33bd52f6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:34.669810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddec76-ee9a-482a-ba55-efb36bfc88ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:34.670718 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddec76-ee9a-482a-ba55-efb36bfc88ef None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:34.671006 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddec76-ee9a-482a-ba55-efb36bfc88ef None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:34.671284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddec76-ee9a-482a-ba55-efb36bfc88ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:34.682543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddec76-ee9a-482a-ba55-efb36bfc88ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4bddec76-ee9a-482a-ba55-efb36bfc88ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:34.690640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:34.691542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:34.692349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 157/313] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:34.698631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57412fb5-a806-45fb-ad1d-caa16dd291b8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:34.805565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57412fb5-a806-45fb-ad1d-caa16dd291b8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:34.840407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d305c1-c9cb-4439-851c-0137ff617feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:34.841086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d305c1-c9cb-4439-851c-0137ff617feb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:34.841368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d305c1-c9cb-4439-851c-0137ff617feb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:34.841645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d305c1-c9cb-4439-851c-0137ff617feb None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:34.842388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d305c1-c9cb-4439-851c-0137ff617feb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:34.842788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d305c1-c9cb-4439-851c-0137ff617feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:34.844189 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d305c1-c9cb-4439-851c-0137ff617feb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:34.853524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 157/314] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:34.859818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddec76-ee9a-482a-ba55-efb36bfc88ef None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:34.967132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddec76-ee9a-482a-ba55-efb36bfc88ef None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:34.970707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:35.004445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9ba95-353d-4fef-b986-fd320d47aefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:35.005667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9ba95-353d-4fef-b986-fd320d47aefc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:35.006141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9ba95-353d-4fef-b986-fd320d47aefc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:35.006706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9ba95-353d-4fef-b986-fd320d47aefc None admin] PATH_INFO: `/v3/users/e6242c39e4f542eda5268fd472ac9380` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:35.008087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9ba95-353d-4fef-b986-fd320d47aefc None admin] RBAC: Authorizing `identity:get_user(user_id=e6242c39e4f542eda5268fd472ac9380)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:35.009774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9ba95-353d-4fef-b986-fd320d47aefc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:35.012166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 158/315] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:34 2022] GET /identity/v3/users/e6242c39e4f542eda5268fd472ac9380 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:35.017527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d305c1-c9cb-4439-851c-0137ff617feb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:35.080248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:35.134079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d305c1-c9cb-4439-851c-0137ff617feb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:35.170826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e99bf-e252-4f62-bf09-4d22e5a0e296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:35.171489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e99bf-e252-4f62-bf09-4d22e5a0e296 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:35.171774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e99bf-e252-4f62-bf09-4d22e5a0e296 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:35.172054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e99bf-e252-4f62-bf09-4d22e5a0e296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:35.246691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e99bf-e252-4f62-bf09-4d22e5a0e296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:35.248217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e99bf-e252-4f62-bf09-4d22e5a0e296 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:35.304502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:35.351874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 158/316] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:35.357990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9ba95-353d-4fef-b986-fd320d47aefc None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:35.455770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9ba95-353d-4fef-b986-fd320d47aefc None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:35.489414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23f5906-0ed4-4891-841b-ecd57574e4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:35.490277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f5906-0ed4-4891-841b-ecd57574e4ae None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:35.490589 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f5906-0ed4-4891-841b-ecd57574e4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:35.490865 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23f5906-0ed4-4891-841b-ecd57574e4ae None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:35.492036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f5906-0ed4-4891-841b-ecd57574e4ae None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:35.493486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23f5906-0ed4-4891-841b-ecd57574e4ae None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:35.495724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 159/317] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:35 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:35.501654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e99bf-e252-4f62-bf09-4d22e5a0e296 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:35.604840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e99bf-e252-4f62-bf09-4d22e5a0e296 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:35.641073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb1f37-b868-473f-a544-aa71994cf54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:35.642081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb1f37-b868-473f-a544-aa71994cf54e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:35.642594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb1f37-b868-473f-a544-aa71994cf54e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:35.643067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb1f37-b868-473f-a544-aa71994cf54e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:35.644061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb1f37-b868-473f-a544-aa71994cf54e None admin] RBAC: Adding query filter params (role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc, scope.project.id=42dcbe790c2a48f2bd52298af1b58573, user.id=e6242c39e4f542eda5268fd472ac9380) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:35.644547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb1f37-b868-473f-a544-aa71994cf54e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:35.646143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb1f37-b868-473f-a544-aa71994cf54e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:35.652869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 159/318] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:50:35 2022] GET /identity/v3/role_assignments?user.id=e6242c39e4f542eda5268fd472ac9380&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:36.246700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cb512-f8cc-44c7-86e1-8657b7df36da None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:36.247077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cb512-f8cc-44c7-86e1-8657b7df36da None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:36.247278 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cb512-f8cc-44c7-86e1-8657b7df36da None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:36.248788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 160/319] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:36.253796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29f9ce-1df7-4131-9ebd-6863f0daef6f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:36.254222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29f9ce-1df7-4131-9ebd-6863f0daef6f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:36.254376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29f9ce-1df7-4131-9ebd-6863f0daef6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:36.274918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3d29f9ce-1df7-4131-9ebd-6863f0daef6f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:36.276177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3d29f9ce-1df7-4131-9ebd-6863f0daef6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:36.293360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:36.354909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 160/320] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:36.378039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cb512-f8cc-44c7-86e1-8657b7df36da None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:36.410077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:36.485727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cb512-f8cc-44c7-86e1-8657b7df36da None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:36.524617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c16d09-84e3-4282-a137-9f609bdf7393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:36.525394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16d09-84e3-4282-a137-9f609bdf7393 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:36.525680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16d09-84e3-4282-a137-9f609bdf7393 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:36.525959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16d09-84e3-4282-a137-9f609bdf7393 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:36.536345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c16d09-84e3-4282-a137-9f609bdf7393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e8c16d09-84e3-4282-a137-9f609bdf7393 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:36.544932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:36.545836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:36.547138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:36.550605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 161/321] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:36.552277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29f9ce-1df7-4131-9ebd-6863f0daef6f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:36.654295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29f9ce-1df7-4131-9ebd-6863f0daef6f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:36.664954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5b921-46f9-4453-8a33-0dca3aac5751 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:36.665247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5b921-46f9-4453-8a33-0dca3aac5751 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:36.665532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5b921-46f9-4453-8a33-0dca3aac5751 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:36.667457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 162/322] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:36.672584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd015413-d398-4a44-9051-8f0bbf757c64 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:36.672873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd015413-d398-4a44-9051-8f0bbf757c64 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:36.673156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd015413-d398-4a44-9051-8f0bbf757c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:36.690301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dd015413-d398-4a44-9051-8f0bbf757c64 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:36.691517 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dd015413-d398-4a44-9051-8f0bbf757c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:36.694183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6515073-4aab-4b02-beef-61a09aacd0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:36.694880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6515073-4aab-4b02-beef-61a09aacd0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:36.695161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6515073-4aab-4b02-beef-61a09aacd0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:36.695434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6515073-4aab-4b02-beef-61a09aacd0d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:36.696168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6515073-4aab-4b02-beef-61a09aacd0d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:36.696471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6515073-4aab-4b02-beef-61a09aacd0d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:36.697885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6515073-4aab-4b02-beef-61a09aacd0d4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:36.705826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 161/323] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:36.707071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62e4fb-c1de-41d8-88a3-105af7a32742 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:36.707359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62e4fb-c1de-41d8-88a3-105af7a32742 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:36.707636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62e4fb-c1de-41d8-88a3-105af7a32742 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:36.708855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 162/324] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:36.711123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62e4fb-c1de-41d8-88a3-105af7a32742 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:36.724289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:36.759162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 163/325] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:36.760456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838444c-6f0a-4cac-9fb0-e38a50446173 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:36.760747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838444c-6f0a-4cac-9fb0-e38a50446173 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:36.761031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838444c-6f0a-4cac-9fb0-e38a50446173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:36.776896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d838444c-6f0a-4cac-9fb0-e38a50446173 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:36.778005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d838444c-6f0a-4cac-9fb0-e38a50446173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:36.794700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:36.816604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62e4fb-c1de-41d8-88a3-105af7a32742 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:36.847786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 164/326] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:36.848630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838444c-6f0a-4cac-9fb0-e38a50446173 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:36.860682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee163f2-2808-4159-a71f-2f349751554d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:36.861566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee163f2-2808-4159-a71f-2f349751554d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:36.861855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee163f2-2808-4159-a71f-2f349751554d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:36.862135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee163f2-2808-4159-a71f-2f349751554d None admin] PATH_INFO: `/v3/users/4f7392485f40434abee66fbdae4c48e4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:36.863656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee163f2-2808-4159-a71f-2f349751554d None admin] RBAC: Authorizing `identity:get_user(user_id=4f7392485f40434abee66fbdae4c48e4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:36.864863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee163f2-2808-4159-a71f-2f349751554d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:36.870598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 163/327] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:36 2022] GET /identity/v3/users/4f7392485f40434abee66fbdae4c48e4 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:36.871539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee163f2-2808-4159-a71f-2f349751554d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:36.945320 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838444c-6f0a-4cac-9fb0-e38a50446173 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:36.946789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:36.968798 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee163f2-2808-4159-a71f-2f349751554d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:36.977562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d81cde-d49e-4fea-9754-ecfb27e71264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:36.978899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d81cde-d49e-4fea-9754-ecfb27e71264 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:36.979183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d81cde-d49e-4fea-9754-ecfb27e71264 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:36.979456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d81cde-d49e-4fea-9754-ecfb27e71264 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:36.989228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d81cde-d49e-4fea-9754-ecfb27e71264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b3d81cde-d49e-4fea-9754-ecfb27e71264 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:36.997050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:36.997921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:36.998751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 165/328] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:36.999429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d81cde-d49e-4fea-9754-ecfb27e71264 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:37.006890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae364ecb-70a2-4511-916c-81aa9eb37483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:37.008076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae364ecb-70a2-4511-916c-81aa9eb37483 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:37.008360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae364ecb-70a2-4511-916c-81aa9eb37483 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:37.008636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae364ecb-70a2-4511-916c-81aa9eb37483 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:37.010157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae364ecb-70a2-4511-916c-81aa9eb37483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:37.011644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae364ecb-70a2-4511-916c-81aa9eb37483 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:37.013222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:37.013490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 164/329] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:37.014293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae364ecb-70a2-4511-916c-81aa9eb37483 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:37.101369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d81cde-d49e-4fea-9754-ecfb27e71264 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:37.114403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae364ecb-70a2-4511-916c-81aa9eb37483 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:37.133756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec12444-abb0-463a-a4bc-e829ed0e4147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:37.134424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec12444-abb0-463a-a4bc-e829ed0e4147 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:37.134741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec12444-abb0-463a-a4bc-e829ed0e4147 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:37.135020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec12444-abb0-463a-a4bc-e829ed0e4147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:37.135840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec12444-abb0-463a-a4bc-e829ed0e4147 None admin] RBAC: Adding query filter params (role.id=d3bdf57150744faf9c5c25c317df48fc, user.id=4f7392485f40434abee66fbdae4c48e4, scope.system=all) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:37.136154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec12444-abb0-463a-a4bc-e829ed0e4147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:37.137866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec12444-abb0-463a-a4bc-e829ed0e4147 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:37.147408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc483b-bd21-4447-b8c4-628629247071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:37.148067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc483b-bd21-4447-b8c4-628629247071 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:37.148351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc483b-bd21-4447-b8c4-628629247071 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:37.148627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc483b-bd21-4447-b8c4-628629247071 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:37.148978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 166/330] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 26 20:50:36 2022] GET /identity/v3/role_assignments?user.id=4f7392485f40434abee66fbdae4c48e4&scope.system=all&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 510 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:37.157056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc483b-bd21-4447-b8c4-628629247071 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:37.157056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc483b-bd21-4447-b8c4-628629247071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:37.157251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec12444-abb0-463a-a4bc-e829ed0e4147 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:37.157321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc483b-bd21-4447-b8c4-628629247071 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:37.165195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 165/331] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:37.171505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc483b-bd21-4447-b8c4-628629247071 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:37.184372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:37.236271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:37.272744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec12444-abb0-463a-a4bc-e829ed0e4147 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:37.288604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc483b-bd21-4447-b8c4-628629247071 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:37.313269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23854c4-0518-4afc-a120-6443e4e373b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:37.313944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23854c4-0518-4afc-a120-6443e4e373b7 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:37.314238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23854c4-0518-4afc-a120-6443e4e373b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:37.314545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23854c4-0518-4afc-a120-6443e4e373b7 None admin] PATH_INFO: `/v3/groups` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:37.315250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23854c4-0518-4afc-a120-6443e4e373b7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:37.316733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23854c4-0518-4afc-a120-6443e4e373b7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:37.328262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3d9c9-2644-49e8-8ab8-e24d50c42628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:37.329153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d9c9-2644-49e8-8ab8-e24d50c42628 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:37.329443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d9c9-2644-49e8-8ab8-e24d50c42628 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:37.329728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3d9c9-2644-49e8-8ab8-e24d50c42628 None admin] PATH_INFO: `/v3/users/811240db838441d4ba462bd6bfae1964` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:37.330965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3d9c9-2644-49e8-8ab8-e24d50c42628 None admin] RBAC: Authorizing `identity:get_user(user_id=811240db838441d4ba462bd6bfae1964)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:37.332415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3d9c9-2644-49e8-8ab8-e24d50c42628 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:37.333606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 167/332] 10.222.0.57 () {60 vars in 1134 bytes} [Thu May 26 20:50:37 2022] POST /identity/v3/groups => generated 226 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:37.334059 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 166/333] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:37 2022] GET /identity/v3/users/811240db838441d4ba462bd6bfae1964 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:37.340346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23854c4-0518-4afc-a120-6443e4e373b7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:37.458687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23854c4-0518-4afc-a120-6443e4e373b7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:37.487906 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:37.494862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fb41a-f37e-4d4e-b79c-6fbf706c5209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:37.495863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fb41a-f37e-4d4e-b79c-6fbf706c5209 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:37.496222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fb41a-f37e-4d4e-b79c-6fbf706c5209 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:37.496659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fb41a-f37e-4d4e-b79c-6fbf706c5209 None admin] PATH_INFO: `/v3/system/users/811240db838441d4ba462bd6bfae1964/roles/23aa53c14bd94d60b9e3ffdcfbecd8dc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:37.498236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fb41a-f37e-4d4e-b79c-6fbf706c5209 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=811240db838441d4ba462bd6bfae1964, role_id=23aa53c14bd94d60b9e3ffdcfbecd8dc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:37.499712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fb41a-f37e-4d4e-b79c-6fbf706c5209 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:37.508252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 168/334] 10.222.0.57 () {58 vars in 1334 bytes} [Thu May 26 20:50:37 2022] PUT /identity/v3/system/users/811240db838441d4ba462bd6bfae1964/roles/23aa53c14bd94d60b9e3ffdcfbecd8dc => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:50:40.131142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d8bde-b99e-48d2-8220-ef5272fb2e69 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.131633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d8bde-b99e-48d2-8220-ef5272fb2e69 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.131716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d8bde-b99e-48d2-8220-ef5272fb2e69 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.133327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 167/335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:40.138638 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6711e16-e380-4bbe-920e-de08a13aaa4e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.139464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6711e16-e380-4bbe-920e-de08a13aaa4e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.139943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6711e16-e380-4bbe-920e-de08a13aaa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.159094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f6711e16-e380-4bbe-920e-de08a13aaa4e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:40.160595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f6711e16-e380-4bbe-920e-de08a13aaa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:40.166876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce38943-07ef-4fed-b166-3cb18eb35b85 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.167453 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce38943-07ef-4fed-b166-3cb18eb35b85 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.167928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce38943-07ef-4fed-b166-3cb18eb35b85 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.169730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 168/336] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:40.175308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28d42b-69c5-4cf6-a9b4-ff34947b473d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.175876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28d42b-69c5-4cf6-a9b4-ff34947b473d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.176355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28d42b-69c5-4cf6-a9b4-ff34947b473d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.202478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8e28d42b-69c5-4cf6-a9b4-ff34947b473d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:40.204350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8e28d42b-69c5-4cf6-a9b4-ff34947b473d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:40.265910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 169/337] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:40.283096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 169/338] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:40.307567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6711e16-e380-4bbe-920e-de08a13aaa4e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:40.308898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28d42b-69c5-4cf6-a9b4-ff34947b473d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:40.326319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:40.455949 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28d42b-69c5-4cf6-a9b4-ff34947b473d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:40.485842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6711e16-e380-4bbe-920e-de08a13aaa4e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:40.493794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745ca41-6cdf-4280-a028-3481ccba08f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:40.494743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745ca41-6cdf-4280-a028-3481ccba08f1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.495119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745ca41-6cdf-4280-a028-3481ccba08f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.495492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745ca41-6cdf-4280-a028-3481ccba08f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.506824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745ca41-6cdf-4280-a028-3481ccba08f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:40.515993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8745ca41-6cdf-4280-a028-3481ccba08f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:40.516553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:40.517423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:40.518220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:40.521680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 170/339] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:40.523594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a751c8-b53e-4e7f-9cfb-23d0175387e1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.524105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a751c8-b53e-4e7f-9cfb-23d0175387e1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.524573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a751c8-b53e-4e7f-9cfb-23d0175387e1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.528297 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 171/340] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:40.535754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60b9fb-da3d-4629-a2b6-2d3b69aa4a06 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.536278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60b9fb-da3d-4629-a2b6-2d3b69aa4a06 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.536671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526b98c-099c-40d0-a1f1-60320ab60ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:40.536971 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60b9fb-da3d-4629-a2b6-2d3b69aa4a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.537483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526b98c-099c-40d0-a1f1-60320ab60ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.537770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526b98c-099c-40d0-a1f1-60320ab60ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.538053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526b98c-099c-40d0-a1f1-60320ab60ee3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.552226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526b98c-099c-40d0-a1f1-60320ab60ee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:40.557275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-dc60b9fb-da3d-4629-a2b6-2d3b69aa4a06 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:40.558715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dc60b9fb-da3d-4629-a2b6-2d3b69aa4a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0526b98c-099c-40d0-a1f1-60320ab60ee3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:40.561520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:40.562379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:40.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 170/341] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:40.563848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526b98c-099c-40d0-a1f1-60320ab60ee3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:40.639075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:40.653951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 172/342] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:40.655257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60b9fb-da3d-4629-a2b6-2d3b69aa4a06 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:40.701479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526b98c-099c-40d0-a1f1-60320ab60ee3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:40.744779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ec002-a70d-442a-879c-8a9ddf2c4087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:40.745376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ec002-a70d-442a-879c-8a9ddf2c4087 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.745663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ec002-a70d-442a-879c-8a9ddf2c4087 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.745943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ec002-a70d-442a-879c-8a9ddf2c4087 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.746787 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ec002-a70d-442a-879c-8a9ddf2c4087 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:40.747043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ec002-a70d-442a-879c-8a9ddf2c4087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:40.748454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ec002-a70d-442a-879c-8a9ddf2c4087 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:40.758425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 171/343] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:40.760227 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142c241-b867-47b7-8b2f-4d4fdbe7b562 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.760519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142c241-b867-47b7-8b2f-4d4fdbe7b562 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.760801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142c241-b867-47b7-8b2f-4d4fdbe7b562 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.762300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 172/344] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:40.767252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142c241-b867-47b7-8b2f-4d4fdbe7b562 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:40.771880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60b9fb-da3d-4629-a2b6-2d3b69aa4a06 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:40.808750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc3c02-8cbf-4559-ab3c-be1fc7804ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:40.809815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3c02-8cbf-4559-ab3c-be1fc7804ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.810325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3c02-8cbf-4559-ab3c-be1fc7804ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.810838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc3c02-8cbf-4559-ab3c-be1fc7804ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.811851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc3c02-8cbf-4559-ab3c-be1fc7804ae0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:40.812379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc3c02-8cbf-4559-ab3c-be1fc7804ae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:40.814026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc3c02-8cbf-4559-ab3c-be1fc7804ae0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:40.825797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 173/345] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:40.831378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc3c02-8cbf-4559-ab3c-be1fc7804ae0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:40.857061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:40.920716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:40.929823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142c241-b867-47b7-8b2f-4d4fdbe7b562 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:40.968519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d76fd4-3edb-42f5-b09a-50eb7a1f00c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:40.968769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc3c02-8cbf-4559-ab3c-be1fc7804ae0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:40.969899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76fd4-3edb-42f5-b09a-50eb7a1f00c2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:40.970736 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76fd4-3edb-42f5-b09a-50eb7a1f00c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:40.972065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76fd4-3edb-42f5-b09a-50eb7a1f00c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:40.987811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d76fd4-3edb-42f5-b09a-50eb7a1f00c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b1d76fd4-3edb-42f5-b09a-50eb7a1f00c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:40.998603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:40.999783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 173/346] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:41.010683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c30505-a31b-4778-81a9-15a1e9df83d6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:41.012664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c30505-a31b-4778-81a9-15a1e9df83d6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:41.012664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c30505-a31b-4778-81a9-15a1e9df83d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:41.029611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df209c2-583c-4028-acc1-ab52de1fecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:41.030754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df209c2-583c-4028-acc1-ab52de1fecc6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:41.031249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df209c2-583c-4028-acc1-ab52de1fecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:41.031918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df209c2-583c-4028-acc1-ab52de1fecc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:41.032039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f7c30505-a31b-4778-81a9-15a1e9df83d6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:41.033328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f7c30505-a31b-4778-81a9-15a1e9df83d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:41.110391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 174/347] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:41.111127 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c30505-a31b-4778-81a9-15a1e9df83d6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:41.121084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df209c2-583c-4028-acc1-ab52de1fecc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:41.123203 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df209c2-583c-4028-acc1-ab52de1fecc6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:41.149742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:41.195645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:41.235351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c30505-a31b-4778-81a9-15a1e9df83d6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:41.245273 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 174/348] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:41.246211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df209c2-583c-4028-acc1-ab52de1fecc6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:41.279701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2c926-db6d-452e-9f4e-b288c798d280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:41.280985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c926-db6d-452e-9f4e-b288c798d280 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:41.281427 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c926-db6d-452e-9f4e-b288c798d280 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:41.281843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2c926-db6d-452e-9f4e-b288c798d280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:41.378044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df209c2-583c-4028-acc1-ab52de1fecc6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:41.394873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2c926-db6d-452e-9f4e-b288c798d280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:41.397314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2c926-db6d-452e-9f4e-b288c798d280 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:41.423575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432bd8c-eb4c-4352-bc3c-bb11e3552f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:41.424257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432bd8c-eb4c-4352-bc3c-bb11e3552f2a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:41.424545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432bd8c-eb4c-4352-bc3c-bb11e3552f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:41.424864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432bd8c-eb4c-4352-bc3c-bb11e3552f2a None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:41.425612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432bd8c-eb4c-4352-bc3c-bb11e3552f2a None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:41.425924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432bd8c-eb4c-4352-bc3c-bb11e3552f2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:41.427378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c432bd8c-eb4c-4352-bc3c-bb11e3552f2a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:41.436477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 175/349] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:41.437124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432bd8c-eb4c-4352-bc3c-bb11e3552f2a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:41.443954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:41.462131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:41.528321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 175/350] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:41.528321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2c926-db6d-452e-9f4e-b288c798d280 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:41.574435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c432bd8c-eb4c-4352-bc3c-bb11e3552f2a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:41.623531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e69093-a4fe-4c1b-8ac6-17b42e650d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:41.624247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e69093-a4fe-4c1b-8ac6-17b42e650d53 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:41.624536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e69093-a4fe-4c1b-8ac6-17b42e650d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:41.624821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e69093-a4fe-4c1b-8ac6-17b42e650d53 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:41.637942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e69093-a4fe-4c1b-8ac6-17b42e650d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d0e69093-a4fe-4c1b-8ac6-17b42e650d53 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:41.645930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:41.646901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 20:50:41.647692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:41.648926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 176/351] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:41.650421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e69093-a4fe-4c1b-8ac6-17b42e650d53 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:41.658766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2c926-db6d-452e-9f4e-b288c798d280 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:41.696485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e557d79-65ef-4443-a82e-46a14c35ca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:41.697896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e557d79-65ef-4443-a82e-46a14c35ca16 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:41.698176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e557d79-65ef-4443-a82e-46a14c35ca16 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:41.698475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e557d79-65ef-4443-a82e-46a14c35ca16 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:41.699675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e557d79-65ef-4443-a82e-46a14c35ca16 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:41.701154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e557d79-65ef-4443-a82e-46a14c35ca16 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:41.703912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 176/352] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:41 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:41.705351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e557d79-65ef-4443-a82e-46a14c35ca16 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:41.710317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:41.756023 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:41.780994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e69093-a4fe-4c1b-8ac6-17b42e650d53 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:41.843013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2116ed3-5331-4bbf-8441-cd346e4c540a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:41.843604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2116ed3-5331-4bbf-8441-cd346e4c540a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:41.843896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2116ed3-5331-4bbf-8441-cd346e4c540a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:41.844005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e557d79-65ef-4443-a82e-46a14c35ca16 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:41.844171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2116ed3-5331-4bbf-8441-cd346e4c540a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:41.890619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6c519-21b8-44a3-b350-4e3d8c5c3ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:41.891549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6c519-21b8-44a3-b350-4e3d8c5c3ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:41.891835 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6c519-21b8-44a3-b350-4e3d8c5c3ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:41.892172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6c519-21b8-44a3-b350-4e3d8c5c3ca7 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:41.893397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6c519-21b8-44a3-b350-4e3d8c5c3ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:41.895432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6c519-21b8-44a3-b350-4e3d8c5c3ca7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:41.897940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 177/353] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:41 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:41.899660 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb742bc-4155-4f65-87ec-5f270e8212bf None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:41.900558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb742bc-4155-4f65-87ec-5f270e8212bf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:41.900848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb742bc-4155-4f65-87ec-5f270e8212bf None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:41.906480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 178/354] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:41.906480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb742bc-4155-4f65-87ec-5f270e8212bf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:41.940208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2116ed3-5331-4bbf-8441-cd346e4c540a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:41.943422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2116ed3-5331-4bbf-8441-cd346e4c540a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:41.995704 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:42.034789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb742bc-4155-4f65-87ec-5f270e8212bf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:42.045017 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:42.070107 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 177/355] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:42.071106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2116ed3-5331-4bbf-8441-cd346e4c540a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:42.078078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b2771-047e-4deb-adc2-b848bb2b9381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:42.078780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2771-047e-4deb-adc2-b848bb2b9381 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.079069 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2771-047e-4deb-adc2-b848bb2b9381 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.079349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2771-047e-4deb-adc2-b848bb2b9381 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.080110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b2771-047e-4deb-adc2-b848bb2b9381 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:42.080427 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b2771-047e-4deb-adc2-b848bb2b9381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:42.081827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b2771-047e-4deb-adc2-b848bb2b9381 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:42.092028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 179/356] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:42.093551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b2771-047e-4deb-adc2-b848bb2b9381 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:42.187478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2116ed3-5331-4bbf-8441-cd346e4c540a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:42.214565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b2771-047e-4deb-adc2-b848bb2b9381 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:42.226853 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325f4b8-0492-465a-96f1-0bb4305ffe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:42.228078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325f4b8-0492-465a-96f1-0bb4305ffe7a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.228632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325f4b8-0492-465a-96f1-0bb4305ffe7a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.228928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325f4b8-0492-465a-96f1-0bb4305ffe7a None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.236367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325f4b8-0492-465a-96f1-0bb4305ffe7a None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:42.237832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325f4b8-0492-465a-96f1-0bb4305ffe7a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:42.239585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 178/357] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:42 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:42.240927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15605e5-f39f-421d-ba8c-d420193c1954 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.241223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15605e5-f39f-421d-ba8c-d420193c1954 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.241506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15605e5-f39f-421d-ba8c-d420193c1954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.260790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae9082-d7fc-4dc9-8016-a51c029e9f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:42.261916 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae9082-d7fc-4dc9-8016-a51c029e9f7a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.262034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a15605e5-f39f-421d-ba8c-d420193c1954 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:42.262211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae9082-d7fc-4dc9-8016-a51c029e9f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.262497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae9082-d7fc-4dc9-8016-a51c029e9f7a None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.263262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a15605e5-f39f-421d-ba8c-d420193c1954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:42.263712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae9082-d7fc-4dc9-8016-a51c029e9f7a None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:42.265334 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae9082-d7fc-4dc9-8016-a51c029e9f7a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:42.269097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 180/358] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:42 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:42.269533 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daad8fa-23c0-4c4d-95cb-f98af45f0891 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.269820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daad8fa-23c0-4c4d-95cb-f98af45f0891 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.270098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daad8fa-23c0-4c4d-95cb-f98af45f0891 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.274583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 181/359] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:42.274583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daad8fa-23c0-4c4d-95cb-f98af45f0891 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:42.278075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:42.312847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:42.348019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 179/360] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:42.348875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15605e5-f39f-421d-ba8c-d420193c1954 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:42.404261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daad8fa-23c0-4c4d-95cb-f98af45f0891 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:42.441175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838f2ce-2622-4660-9d7b-85131f5b5cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:42.442051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838f2ce-2622-4660-9d7b-85131f5b5cbe None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.442339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838f2ce-2622-4660-9d7b-85131f5b5cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.442657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a838f2ce-2622-4660-9d7b-85131f5b5cbe None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.443877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838f2ce-2622-4660-9d7b-85131f5b5cbe None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:42.445332 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a838f2ce-2622-4660-9d7b-85131f5b5cbe None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:42.447618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 182/361] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:42 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:42.448544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838f2ce-2622-4660-9d7b-85131f5b5cbe None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:42.467209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15605e5-f39f-421d-ba8c-d420193c1954 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:42.510313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53473b-f471-4dda-a5ff-13931bfcf361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:42.511225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53473b-f471-4dda-a5ff-13931bfcf361 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.511508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53473b-f471-4dda-a5ff-13931bfcf361 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.512038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53473b-f471-4dda-a5ff-13931bfcf361 None admin] PATH_INFO: `/v3/users/811240db838441d4ba462bd6bfae1964` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.513220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53473b-f471-4dda-a5ff-13931bfcf361 None admin] RBAC: Authorizing `identity:get_user(user_id=811240db838441d4ba462bd6bfae1964)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:42.514719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53473b-f471-4dda-a5ff-13931bfcf361 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:42.517038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 180/362] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:50:42 2022] GET /identity/v3/users/811240db838441d4ba462bd6bfae1964 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:42.517164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53473b-f471-4dda-a5ff-13931bfcf361 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:42.530934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:42.568562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a838f2ce-2622-4660-9d7b-85131f5b5cbe None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:42.573692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:42.610438 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c96037-a4a8-4a27-98bf-d3c71e88a3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:42.612192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96037-a4a8-4a27-98bf-d3c71e88a3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.612192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96037-a4a8-4a27-98bf-d3c71e88a3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.612192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c96037-a4a8-4a27-98bf-d3c71e88a3c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.612418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c96037-a4a8-4a27-98bf-d3c71e88a3c5 None admin] RBAC: Adding query filter params (group.id=8d5629ebc0bc41cab9b6560424962390, role.id=d3bdf57150744faf9c5c25c317df48fc, scope.project.id=bb614e1d464d4df1959debd82ba5049c) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:42.613530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c96037-a4a8-4a27-98bf-d3c71e88a3c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:42.614116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c96037-a4a8-4a27-98bf-d3c71e88a3c5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:42.621405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 183/363] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 26 20:50:42 2022] GET /identity/v3/role_assignments?group.id=8d5629ebc0bc41cab9b6560424962390&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:42.626747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4926e9b-8f7c-428f-bee6-bd7971781f0d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.627099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4926e9b-8f7c-428f-bee6-bd7971781f0d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.627389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4926e9b-8f7c-428f-bee6-bd7971781f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.642322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53473b-f471-4dda-a5ff-13931bfcf361 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:42.649020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e4926e9b-8f7c-428f-bee6-bd7971781f0d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:42.650304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e4926e9b-8f7c-428f-bee6-bd7971781f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:42.686495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e91a1-2413-437c-a4a6-e02ed6fff706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:42.687150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e91a1-2413-437c-a4a6-e02ed6fff706 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.687483 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e91a1-2413-437c-a4a6-e02ed6fff706 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.687703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e91a1-2413-437c-a4a6-e02ed6fff706 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.689160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e91a1-2413-437c-a4a6-e02ed6fff706 None admin] RBAC: Adding query filter params (group.id=8d5629ebc0bc41cab9b6560424962390, role.id=fa67cb2f211d433097f606ff029f863e, scope.project.id=bb614e1d464d4df1959debd82ba5049c) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:42.689160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e91a1-2413-437c-a4a6-e02ed6fff706 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:42.691877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e91a1-2413-437c-a4a6-e02ed6fff706 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:42.698884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 181/364] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 26 20:50:42 2022] GET /identity/v3/role_assignments?group.id=8d5629ebc0bc41cab9b6560424962390&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=fa67cb2f211d433097f606ff029f863e => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:42.699565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e91a1-2413-437c-a4a6-e02ed6fff706 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:42.749151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 184/365] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:42.751237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4926e9b-8f7c-428f-bee6-bd7971781f0d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:42.858724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e91a1-2413-437c-a4a6-e02ed6fff706 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:42.861062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:42.872145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:42.909088 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6c353-9357-4d15-ac2c-2de77eb13523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:42.909924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6c353-9357-4d15-ac2c-2de77eb13523 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.910205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6c353-9357-4d15-ac2c-2de77eb13523 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.910972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6c353-9357-4d15-ac2c-2de77eb13523 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.919170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4926e9b-8f7c-428f-bee6-bd7971781f0d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:42.923463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6c353-9357-4d15-ac2c-2de77eb13523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-13d6c353-9357-4d15-ac2c-2de77eb13523 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:42.932465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:42.933400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:42.934486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 182/366] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:42.935365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6c353-9357-4d15-ac2c-2de77eb13523 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:42.969278 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49368ad3-a79c-48b9-8647-1c29179e93ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:42.970738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368ad3-a79c-48b9-8647-1c29179e93ff None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:42.971161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368ad3-a79c-48b9-8647-1c29179e93ff None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:42.971596 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368ad3-a79c-48b9-8647-1c29179e93ff None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:42.973200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49368ad3-a79c-48b9-8647-1c29179e93ff None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:42.975556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49368ad3-a79c-48b9-8647-1c29179e93ff None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:42.977646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 185/367] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:42 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:42.979513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49368ad3-a79c-48b9-8647-1c29179e93ff None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:43.056483 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6c353-9357-4d15-ac2c-2de77eb13523 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:43.092738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df0efb-3fc3-489e-8748-c5d6010005e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:43.093510 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49368ad3-a79c-48b9-8647-1c29179e93ff None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:43.094844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df0efb-3fc3-489e-8748-c5d6010005e5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:43.095282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df0efb-3fc3-489e-8748-c5d6010005e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:43.095699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df0efb-3fc3-489e-8748-c5d6010005e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:43.096678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df0efb-3fc3-489e-8748-c5d6010005e5 None admin] RBAC: Adding query filter params (role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc, user.id=811240db838441d4ba462bd6bfae1964, scope.system=all) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:43.097223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df0efb-3fc3-489e-8748-c5d6010005e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:43.098917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02df0efb-3fc3-489e-8748-c5d6010005e5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:43.112775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 183/368] 10.222.0.57 () {56 vars in 1308 bytes} [Thu May 26 20:50:42 2022] GET /identity/v3/role_assignments?user.id=811240db838441d4ba462bd6bfae1964&scope.system=all&role.id=23aa53c14bd94d60b9e3ffdcfbecd8dc => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:43.114541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df0efb-3fc3-489e-8748-c5d6010005e5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:43.127664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:43.133280 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0d8eb-eb27-45e5-80de-bbfa30f0c7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:43.133824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0d8eb-eb27-45e5-80de-bbfa30f0c7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:43.134117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0d8eb-eb27-45e5-80de-bbfa30f0c7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:43.134524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0d8eb-eb27-45e5-80de-bbfa30f0c7c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:43.147706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0d8eb-eb27-45e5-80de-bbfa30f0c7c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:43.148468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fef0d8eb-eb27-45e5-80de-bbfa30f0c7c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:43.157888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:43.158828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:43.160126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 186/369] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:50:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:43.160975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0d8eb-eb27-45e5-80de-bbfa30f0c7c6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:43.239632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df0efb-3fc3-489e-8748-c5d6010005e5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:43.273852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0d8eb-eb27-45e5-80de-bbfa30f0c7c6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:43.279881 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6306a6-6b8d-4f92-9ed6-a43ba7b40dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:43.280612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6306a6-6b8d-4f92-9ed6-a43ba7b40dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:43.280898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6306a6-6b8d-4f92-9ed6-a43ba7b40dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:43.281176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6306a6-6b8d-4f92-9ed6-a43ba7b40dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:43.281934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6306a6-6b8d-4f92-9ed6-a43ba7b40dd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:43.282246 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6306a6-6b8d-4f92-9ed6-a43ba7b40dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:43.283749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6306a6-6b8d-4f92-9ed6-a43ba7b40dd3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:43.293458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 184/370] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:43.294375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6306a6-6b8d-4f92-9ed6-a43ba7b40dd3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:43.312415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b14adc-ab0b-48f4-90a4-1a801682315b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:43.313085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b14adc-ab0b-48f4-90a4-1a801682315b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:43.313373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b14adc-ab0b-48f4-90a4-1a801682315b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:43.313652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b14adc-ab0b-48f4-90a4-1a801682315b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:43.314515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b14adc-ab0b-48f4-90a4-1a801682315b None admin] RBAC: Adding query filter params (group.id=8d5629ebc0bc41cab9b6560424962390, role.id=d3bdf57150744faf9c5c25c317df48fc, scope.project.id=42dcbe790c2a48f2bd52298af1b58573) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:43.314835 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b14adc-ab0b-48f4-90a4-1a801682315b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:43.316303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b14adc-ab0b-48f4-90a4-1a801682315b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:43.323440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 187/371] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 26 20:50:43 2022] GET /identity/v3/role_assignments?group.id=8d5629ebc0bc41cab9b6560424962390&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:43.324348 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b14adc-ab0b-48f4-90a4-1a801682315b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:43.390485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:43.395768 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:43.415930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6306a6-6b8d-4f92-9ed6-a43ba7b40dd3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:43.454962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646dd9c3-93e4-427a-97f7-6c1634549cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:43.455597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b14adc-ab0b-48f4-90a4-1a801682315b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:43.455715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dd9c3-93e4-427a-97f7-6c1634549cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:43.455960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dd9c3-93e4-427a-97f7-6c1634549cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:43.456182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646dd9c3-93e4-427a-97f7-6c1634549cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:43.456935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646dd9c3-93e4-427a-97f7-6c1634549cb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:43.457356 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646dd9c3-93e4-427a-97f7-6c1634549cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:43.458914 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646dd9c3-93e4-427a-97f7-6c1634549cb9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:43.470021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 185/372] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:43.475195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646dd9c3-93e4-427a-97f7-6c1634549cb9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:43.502683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde453d7-f375-4387-b304-31d2bbf674a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:43.503359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde453d7-f375-4387-b304-31d2bbf674a2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:43.503650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde453d7-f375-4387-b304-31d2bbf674a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:43.503934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde453d7-f375-4387-b304-31d2bbf674a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:43.606660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646dd9c3-93e4-427a-97f7-6c1634549cb9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:43.607319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde453d7-f375-4387-b304-31d2bbf674a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:43.608912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde453d7-f375-4387-b304-31d2bbf674a2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:43.647757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42b20f-199a-4561-8318-075dc291aed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:43.648374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b20f-199a-4561-8318-075dc291aed1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:43.648622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b20f-199a-4561-8318-075dc291aed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:43.648902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42b20f-199a-4561-8318-075dc291aed1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:43.673741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:43.688241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:43.732890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42b20f-199a-4561-8318-075dc291aed1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:43.734482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42b20f-199a-4561-8318-075dc291aed1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:43.754185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 188/373] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:43.759878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde453d7-f375-4387-b304-31d2bbf674a2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:43.845934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 186/374] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:43.852324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42b20f-199a-4561-8318-075dc291aed1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:43.909325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:43.958431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde453d7-f375-4387-b304-31d2bbf674a2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:43.965144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42b20f-199a-4561-8318-075dc291aed1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:44.004255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d4331-6adc-48b8-aa34-1cf2a0619f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:44.005129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d4331-6adc-48b8-aa34-1cf2a0619f74 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:44.005407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d4331-6adc-48b8-aa34-1cf2a0619f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:44.005681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d4331-6adc-48b8-aa34-1cf2a0619f74 None admin] PATH_INFO: `/v3/projects/116e4cb137e24acda75f964e89e322c5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:44.007644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d4331-6adc-48b8-aa34-1cf2a0619f74 None admin] RBAC: Authorizing `identity:get_project(project_id=116e4cb137e24acda75f964e89e322c5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:44.009113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d4331-6adc-48b8-aa34-1cf2a0619f74 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:44.011056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 187/375] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:43 2022] GET /identity/v3/projects/116e4cb137e24acda75f964e89e322c5 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:44.019986 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d4331-6adc-48b8-aa34-1cf2a0619f74 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:44.026426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea15c-cfd6-4cec-a05c-6260115d3124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:44.027306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea15c-cfd6-4cec-a05c-6260115d3124 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:44.027584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea15c-cfd6-4cec-a05c-6260115d3124 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:44.027856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ea15c-cfd6-4cec-a05c-6260115d3124 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:44.029711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ea15c-cfd6-4cec-a05c-6260115d3124 None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:44.037122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ea15c-cfd6-4cec-a05c-6260115d3124 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:44.039079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 189/376] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:43 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:44.045124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea15c-cfd6-4cec-a05c-6260115d3124 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:44.093475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:44.225703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d4331-6adc-48b8-aa34-1cf2a0619f74 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:44.246967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:44.252763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ea15c-cfd6-4cec-a05c-6260115d3124 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:44.270420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69991f38-ff97-47bb-8280-9535100a87b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:44.271556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69991f38-ff97-47bb-8280-9535100a87b7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:44.271992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69991f38-ff97-47bb-8280-9535100a87b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:44.272406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69991f38-ff97-47bb-8280-9535100a87b7 None admin] PATH_INFO: `/v3/groups/1e126b8a6f9743459e30e085be179a4f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:44.277919 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69991f38-ff97-47bb-8280-9535100a87b7 None admin] RBAC: Authorizing `identity:get_group(group_id=1e126b8a6f9743459e30e085be179a4f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:44.279554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69991f38-ff97-47bb-8280-9535100a87b7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:44.286226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 188/377] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:44 2022] GET /identity/v3/groups/1e126b8a6f9743459e30e085be179a4f => generated 229 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:44.287538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69991f38-ff97-47bb-8280-9535100a87b7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:44.310274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7654b85-d092-4125-980a-326e9766b027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:44.311342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7654b85-d092-4125-980a-326e9766b027 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:44.311723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7654b85-d092-4125-980a-326e9766b027 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:44.312098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7654b85-d092-4125-980a-326e9766b027 None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:44.313380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7654b85-d092-4125-980a-326e9766b027 None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:44.315110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7654b85-d092-4125-980a-326e9766b027 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:44.316940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 190/378] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:44 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:44.322791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7654b85-d092-4125-980a-326e9766b027 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:44.411597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69991f38-ff97-47bb-8280-9535100a87b7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:44.446023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:44.462095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7654b85-d092-4125-980a-326e9766b027 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:44.468189 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364308f-ada8-45bb-b673-97f29544a275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:44.469019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364308f-ada8-45bb-b673-97f29544a275 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:44.469666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364308f-ada8-45bb-b673-97f29544a275 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:44.470106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364308f-ada8-45bb-b673-97f29544a275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:44.472826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364308f-ada8-45bb-b673-97f29544a275 None admin] RBAC: Adding query filter params (group.id=1e126b8a6f9743459e30e085be179a4f, role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=116e4cb137e24acda75f964e89e322c5) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:44.476228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364308f-ada8-45bb-b673-97f29544a275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:44.477825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2364308f-ada8-45bb-b673-97f29544a275 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:44.487025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 189/379] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 26 20:50:44 2022] GET /identity/v3/role_assignments?group.id=1e126b8a6f9743459e30e085be179a4f&scope.project.id=116e4cb137e24acda75f964e89e322c5&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:44.516601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f23400-4135-410f-bccb-fe0fabe99e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:44.517496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f23400-4135-410f-bccb-fe0fabe99e16 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:44.517938 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f23400-4135-410f-bccb-fe0fabe99e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:44.518352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f23400-4135-410f-bccb-fe0fabe99e16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:44.519392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f23400-4135-410f-bccb-fe0fabe99e16 None admin] RBAC: Adding query filter params (group.id=8d5629ebc0bc41cab9b6560424962390, role.id=fa67cb2f211d433097f606ff029f863e, scope.project.id=42dcbe790c2a48f2bd52298af1b58573) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:44.519866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f23400-4135-410f-bccb-fe0fabe99e16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:44.521523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f23400-4135-410f-bccb-fe0fabe99e16 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:44.530290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 191/380] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 26 20:50:44 2022] GET /identity/v3/role_assignments?group.id=8d5629ebc0bc41cab9b6560424962390&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=fa67cb2f211d433097f606ff029f863e => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:45.548947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab870454-5e37-41c5-bbcf-7b94386ebbd1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:45.549492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab870454-5e37-41c5-bbcf-7b94386ebbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:45.549492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab870454-5e37-41c5-bbcf-7b94386ebbd1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:45.551153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 190/381] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:45.556565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83951a17-df94-4f32-81fa-011c09486ee6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:45.557405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83951a17-df94-4f32-81fa-011c09486ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:45.557906 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83951a17-df94-4f32-81fa-011c09486ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:45.575594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-83951a17-df94-4f32-81fa-011c09486ee6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:45.577070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-83951a17-df94-4f32-81fa-011c09486ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:45.646833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 192/382] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:45.669903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab870454-5e37-41c5-bbcf-7b94386ebbd1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:45.738998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:45.823686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab870454-5e37-41c5-bbcf-7b94386ebbd1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:45.875278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fd845-5ecf-403b-aef5-d5df56258c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:45.876096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fd845-5ecf-403b-aef5-d5df56258c64 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:45.876381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fd845-5ecf-403b-aef5-d5df56258c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:45.876656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9fd845-5ecf-403b-aef5-d5df56258c64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:45.878709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fbc68-78c5-4ab0-a805-6148185e98ea None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:45.879008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fbc68-78c5-4ab0-a805-6148185e98ea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:45.879292 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fbc68-78c5-4ab0-a805-6148185e98ea None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:45.880999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 193/383] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:45.888717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ec8df-a161-42d3-bcc7-ecb60ced582e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:45.889013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ec8df-a161-42d3-bcc7-ecb60ced582e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:45.889291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ec8df-a161-42d3-bcc7-ecb60ced582e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:45.892570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9fd845-5ecf-403b-aef5-d5df56258c64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-fe9fd845-5ecf-403b-aef5-d5df56258c64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:45.900977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:45.901932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:45.902790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 191/384] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:45.908818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fd845-5ecf-403b-aef5-d5df56258c64 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:45.929483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-773ec8df-a161-42d3-bcc7-ecb60ced582e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:45.930839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-773ec8df-a161-42d3-bcc7-ecb60ced582e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:46.058738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9fd845-5ecf-403b-aef5-d5df56258c64 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:46.061469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:46.106521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 194/385] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:46.116273 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6412d56-295b-4e24-aa7e-ca461b216d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:46.116957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6412d56-295b-4e24-aa7e-ca461b216d24 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:46.117243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6412d56-295b-4e24-aa7e-ca461b216d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:46.117527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6412d56-295b-4e24-aa7e-ca461b216d24 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:46.118280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6412d56-295b-4e24-aa7e-ca461b216d24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:46.119154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6412d56-295b-4e24-aa7e-ca461b216d24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:46.120558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6412d56-295b-4e24-aa7e-ca461b216d24 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:46.129132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 192/386] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:46.135370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ec8df-a161-42d3-bcc7-ecb60ced582e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:46.145563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6412d56-295b-4e24-aa7e-ca461b216d24 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:46.174143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:46.279397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773ec8df-a161-42d3-bcc7-ecb60ced582e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:46.307964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6412d56-295b-4e24-aa7e-ca461b216d24 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:46.315780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87281e1-64f7-4ab8-9959-fc1999b54e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:46.316641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87281e1-64f7-4ab8-9959-fc1999b54e43 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:46.316929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87281e1-64f7-4ab8-9959-fc1999b54e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:46.317210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87281e1-64f7-4ab8-9959-fc1999b54e43 None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:46.318387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87281e1-64f7-4ab8-9959-fc1999b54e43 None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:46.319998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87281e1-64f7-4ab8-9959-fc1999b54e43 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:46.326142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 195/387] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:46 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:46.327436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd52ed-15c7-43c6-8cef-a06c210c70c2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:46.327724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd52ed-15c7-43c6-8cef-a06c210c70c2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:46.328004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd52ed-15c7-43c6-8cef-a06c210c70c2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:46.329246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 196/388] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:46.334272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd52ed-15c7-43c6-8cef-a06c210c70c2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:46.347995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c02ab-5e63-4703-91bd-002d20db48b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:46.348964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c02ab-5e63-4703-91bd-002d20db48b6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:46.349636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c02ab-5e63-4703-91bd-002d20db48b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:46.349927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c02ab-5e63-4703-91bd-002d20db48b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:46.367957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89c02ab-5e63-4703-91bd-002d20db48b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c89c02ab-5e63-4703-91bd-002d20db48b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:46.377014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:46.377924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:46.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 193/389] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:46.380154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575c04f-86fa-46e3-9e51-0089faeecdd4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:46.380154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575c04f-86fa-46e3-9e51-0089faeecdd4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:46.380154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575c04f-86fa-46e3-9e51-0089faeecdd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:46.398047 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a575c04f-86fa-46e3-9e51-0089faeecdd4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:46.399259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a575c04f-86fa-46e3-9e51-0089faeecdd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:46.422847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:46.444873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:46.465734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd52ed-15c7-43c6-8cef-a06c210c70c2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:46.489846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 194/390] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:46.491018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575c04f-86fa-46e3-9e51-0089faeecdd4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:46.519193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276870c2-460d-4440-a8c5-c51149c18b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:46.520058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276870c2-460d-4440-a8c5-c51149c18b17 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:46.520341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276870c2-460d-4440-a8c5-c51149c18b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:46.520618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276870c2-460d-4440-a8c5-c51149c18b17 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:46.527210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276870c2-460d-4440-a8c5-c51149c18b17 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:46.528806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276870c2-460d-4440-a8c5-c51149c18b17 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:46.530812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 197/391] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:46 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:46.537516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276870c2-460d-4440-a8c5-c51149c18b17 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:46.600686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575c04f-86fa-46e3-9e51-0089faeecdd4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:46.638791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fd7b0-3cca-4781-87cc-e2d846f529c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:46.639874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fd7b0-3cca-4781-87cc-e2d846f529c8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:46.640372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fd7b0-3cca-4781-87cc-e2d846f529c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:46.640910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fd7b0-3cca-4781-87cc-e2d846f529c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:46.641880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fd7b0-3cca-4781-87cc-e2d846f529c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:46.642416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fd7b0-3cca-4781-87cc-e2d846f529c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:46.644021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fd7b0-3cca-4781-87cc-e2d846f529c8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:46.644204 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276870c2-460d-4440-a8c5-c51149c18b17 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:46.654034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 195/392] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:46.655217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fd7b0-3cca-4781-87cc-e2d846f529c8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:46.675537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:46.682925 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4fa85-0059-4f0d-9531-5aa446dab554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:46.683882 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4fa85-0059-4f0d-9531-5aa446dab554 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:46.684090 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4fa85-0059-4f0d-9531-5aa446dab554 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:46.684367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4fa85-0059-4f0d-9531-5aa446dab554 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c/groups/8d5629ebc0bc41cab9b6560424962390/roles/fa67cb2f211d433097f606ff029f863e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:46.686984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4fa85-0059-4f0d-9531-5aa446dab554 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb614e1d464d4df1959debd82ba5049c, group_id=8d5629ebc0bc41cab9b6560424962390, role_id=fa67cb2f211d433097f606ff029f863e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:46.688535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4fa85-0059-4f0d-9531-5aa446dab554 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:46.713993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 198/393] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 26 20:50:46 2022] PUT /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c/groups/8d5629ebc0bc41cab9b6560424962390/roles/fa67cb2f211d433097f606ff029f863e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:50:46.715532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4fa85-0059-4f0d-9531-5aa446dab554 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:46.746330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:46.833212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fd7b0-3cca-4781-87cc-e2d846f529c8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:46.906139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6961a6-8458-48e9-b125-8b4a2c2299d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:46.906906 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6961a6-8458-48e9-b125-8b4a2c2299d7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:46.907199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6961a6-8458-48e9-b125-8b4a2c2299d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:46.907483 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6961a6-8458-48e9-b125-8b4a2c2299d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:46.924554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4fa85-0059-4f0d-9531-5aa446dab554 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:46.933124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6961a6-8458-48e9-b125-8b4a2c2299d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9a6961a6-8458-48e9-b125-8b4a2c2299d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:46.950431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:46.951421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:46.952224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:46.955367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 196/394] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:46.961274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6961a6-8458-48e9-b125-8b4a2c2299d7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:46.979047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c9d71-6de7-4ba7-9fca-be5393de797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:46.979911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c9d71-6de7-4ba7-9fca-be5393de797e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:46.980196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c9d71-6de7-4ba7-9fca-be5393de797e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:46.980552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c9d71-6de7-4ba7-9fca-be5393de797e None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:46.981660 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c9d71-6de7-4ba7-9fca-be5393de797e None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:46.983214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c9d71-6de7-4ba7-9fca-be5393de797e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:46.984922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 199/395] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:46 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:46.990205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c9d71-6de7-4ba7-9fca-be5393de797e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:47.048720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:47.075420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6961a6-8458-48e9-b125-8b4a2c2299d7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:47.076530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:47.101209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c9d71-6de7-4ba7-9fca-be5393de797e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:47.111874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d7435-265e-49d9-8b83-d7b0f8265e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:47.112407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d7435-265e-49d9-8b83-d7b0f8265e7b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.112796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d7435-265e-49d9-8b83-d7b0f8265e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.113175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d7435-265e-49d9-8b83-d7b0f8265e7b None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.114152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d7435-265e-49d9-8b83-d7b0f8265e7b None admin] RBAC: Adding query filter params (name=member) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:47.114419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d7435-265e-49d9-8b83-d7b0f8265e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:47.115907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d7435-265e-49d9-8b83-d7b0f8265e7b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:47.129828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 197/396] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:47.131529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7536f78-7325-478a-a8f5-84be1335e76f None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.131934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7536f78-7325-478a-a8f5-84be1335e76f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.132315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7536f78-7325-478a-a8f5-84be1335e76f None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.134336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 198/397] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:47.138811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7536f78-7325-478a-a8f5-84be1335e76f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:47.147009 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7567f-81de-45a1-ab99-6b76a84a34ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:47.148261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7567f-81de-45a1-ab99-6b76a84a34ca None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.148694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7567f-81de-45a1-ab99-6b76a84a34ca None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.149131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7567f-81de-45a1-ab99-6b76a84a34ca None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.150773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7567f-81de-45a1-ab99-6b76a84a34ca None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:47.152516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7567f-81de-45a1-ab99-6b76a84a34ca None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:47.154982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 200/398] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:46 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:47.156652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f6b95-401a-4572-85a1-a52825966265 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.157124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f6b95-401a-4572-85a1-a52825966265 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.157567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f6b95-401a-4572-85a1-a52825966265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.176377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-942f6b95-401a-4572-85a1-a52825966265 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:47.177795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-942f6b95-401a-4572-85a1-a52825966265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:47.248633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 201/399] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:47.254501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7536f78-7325-478a-a8f5-84be1335e76f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:47.255543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f6b95-401a-4572-85a1-a52825966265 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:47.289060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dcc1a-7f83-4d9d-a30f-68ac0f149f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:47.290087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dcc1a-7f83-4d9d-a30f-68ac0f149f4c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.291004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dcc1a-7f83-4d9d-a30f-68ac0f149f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.291381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dcc1a-7f83-4d9d-a30f-68ac0f149f4c None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.292719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dcc1a-7f83-4d9d-a30f-68ac0f149f4c None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:47.294481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dcc1a-7f83-4d9d-a30f-68ac0f149f4c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:47.296305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 199/400] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:47 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:47.302846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dcc1a-7f83-4d9d-a30f-68ac0f149f4c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:47.313553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:47.360869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:47.383659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f6b95-401a-4572-85a1-a52825966265 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:47.413413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dcc1a-7f83-4d9d-a30f-68ac0f149f4c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:47.427807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c1fc5-af1f-4bc9-af77-8fb4c581c375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:47.428819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c1fc5-af1f-4bc9-af77-8fb4c581c375 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.429100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c1fc5-af1f-4bc9-af77-8fb4c581c375 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.429387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c1fc5-af1f-4bc9-af77-8fb4c581c375 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c/groups/8d5629ebc0bc41cab9b6560424962390/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.431439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c1fc5-af1f-4bc9-af77-8fb4c581c375 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb614e1d464d4df1959debd82ba5049c, group_id=8d5629ebc0bc41cab9b6560424962390, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:47.432896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c1fc5-af1f-4bc9-af77-8fb4c581c375 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:47.447152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 202/401] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 26 20:50:47 2022] PUT /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c/groups/8d5629ebc0bc41cab9b6560424962390/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:50:47.451428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d837c9-57bc-4ee2-827f-53cf22bb070e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:47.452324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d837c9-57bc-4ee2-827f-53cf22bb070e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.452609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d837c9-57bc-4ee2-827f-53cf22bb070e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.452890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d837c9-57bc-4ee2-827f-53cf22bb070e None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.454174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d837c9-57bc-4ee2-827f-53cf22bb070e None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:47.455730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d837c9-57bc-4ee2-827f-53cf22bb070e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:47.457724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 200/402] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:47 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:47.466683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94b1da-d06e-45fd-916f-5349766e7024 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.466683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94b1da-d06e-45fd-916f-5349766e7024 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.466931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c1fc5-af1f-4bc9-af77-8fb4c581c375 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:47.467040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94b1da-d06e-45fd-916f-5349766e7024 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.467040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 201/403] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:47.467040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94b1da-d06e-45fd-916f-5349766e7024 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:47.569257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:47.599568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94b1da-d06e-45fd-916f-5349766e7024 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:47.619118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c1fc5-af1f-4bc9-af77-8fb4c581c375 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:47.644106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7739f-2109-44db-b2b3-a01fbbfc1f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:47.645274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7739f-2109-44db-b2b3-a01fbbfc1f49 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.645563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7739f-2109-44db-b2b3-a01fbbfc1f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.645993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7739f-2109-44db-b2b3-a01fbbfc1f49 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573/groups/8d5629ebc0bc41cab9b6560424962390/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.650511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7739f-2109-44db-b2b3-a01fbbfc1f49 None admin] RBAC: Authorizing `identity:create_grant(project_id=42dcbe790c2a48f2bd52298af1b58573, group_id=8d5629ebc0bc41cab9b6560424962390, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:47.651958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7739f-2109-44db-b2b3-a01fbbfc1f49 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:47.664502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54787566-3ddf-4033-b081-479a938ce7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:47.665243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54787566-3ddf-4033-b081-479a938ce7ce None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.666041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54787566-3ddf-4033-b081-479a938ce7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.666316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54787566-3ddf-4033-b081-479a938ce7ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.671452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 202/404] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 26 20:50:47 2022] PUT /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573/groups/8d5629ebc0bc41cab9b6560424962390/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:50:47.672925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378a44a-9c8e-4ddb-8d9b-1d7af59e5beb None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.673400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378a44a-9c8e-4ddb-8d9b-1d7af59e5beb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.673881 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378a44a-9c8e-4ddb-8d9b-1d7af59e5beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.680121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:47.689578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54787566-3ddf-4033-b081-479a938ce7ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:47.695087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f378a44a-9c8e-4ddb-8d9b-1d7af59e5beb None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:47.697997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f378a44a-9c8e-4ddb-8d9b-1d7af59e5beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-54787566-3ddf-4033-b081-479a938ce7ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:47.701046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:47.701939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:47.702806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:47.708265 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 203/405] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:50:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:47.714263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54787566-3ddf-4033-b081-479a938ce7ce None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:47.801498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 203/406] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:47.841879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378a44a-9c8e-4ddb-8d9b-1d7af59e5beb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:47.926010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54787566-3ddf-4033-b081-479a938ce7ce None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:47.938209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:47.979262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f9eae-7c05-493a-848d-7796cce3b770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:47.979937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f9eae-7c05-493a-848d-7796cce3b770 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:47.980219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f9eae-7c05-493a-848d-7796cce3b770 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:47.980493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67f9eae-7c05-493a-848d-7796cce3b770 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:47.981666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f9eae-7c05-493a-848d-7796cce3b770 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:47.982128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f9eae-7c05-493a-848d-7796cce3b770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:47.983666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67f9eae-7c05-493a-848d-7796cce3b770 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:47.984442 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378a44a-9c8e-4ddb-8d9b-1d7af59e5beb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:47.993821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 204/407] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:47.998985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f9eae-7c05-493a-848d-7796cce3b770 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:48.041682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393fe6f-95b2-4f06-8d32-74d1151d143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:48.043635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393fe6f-95b2-4f06-8d32-74d1151d143e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:48.044250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393fe6f-95b2-4f06-8d32-74d1151d143e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:48.044764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393fe6f-95b2-4f06-8d32-74d1151d143e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:48.068261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393fe6f-95b2-4f06-8d32-74d1151d143e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:48.083419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2393fe6f-95b2-4f06-8d32-74d1151d143e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:48.084780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:48.085711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:48.086870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 204/408] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:48.093098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393fe6f-95b2-4f06-8d32-74d1151d143e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:48.128672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67f9eae-7c05-493a-848d-7796cce3b770 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:48.169042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b87e83-a402-4f4d-a3a0-87f40f78f3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:48.169913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b87e83-a402-4f4d-a3a0-87f40f78f3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:48.170193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b87e83-a402-4f4d-a3a0-87f40f78f3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:48.170492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b87e83-a402-4f4d-a3a0-87f40f78f3c2 None admin] PATH_INFO: `/v3/groups/1e126b8a6f9743459e30e085be179a4f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:48.171799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b87e83-a402-4f4d-a3a0-87f40f78f3c2 None admin] RBAC: Authorizing `identity:get_group(group_id=1e126b8a6f9743459e30e085be179a4f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:48.174129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b87e83-a402-4f4d-a3a0-87f40f78f3c2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:48.180223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 205/409] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:47 2022] GET /identity/v3/groups/1e126b8a6f9743459e30e085be179a4f => generated 229 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:48.188702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b87e83-a402-4f4d-a3a0-87f40f78f3c2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:48.295979 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b87e83-a402-4f4d-a3a0-87f40f78f3c2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:48.327082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:48.339446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782350db-37d0-49d0-bbde-cd9cc2a82917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:48.341651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782350db-37d0-49d0-bbde-cd9cc2a82917 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:48.343943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393fe6f-95b2-4f06-8d32-74d1151d143e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:48.346661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782350db-37d0-49d0-bbde-cd9cc2a82917 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:48.347169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782350db-37d0-49d0-bbde-cd9cc2a82917 None admin] PATH_INFO: `/v3/projects/116e4cb137e24acda75f964e89e322c5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:48.348691 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782350db-37d0-49d0-bbde-cd9cc2a82917 None admin] RBAC: Authorizing `identity:get_project(project_id=116e4cb137e24acda75f964e89e322c5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:48.351206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782350db-37d0-49d0-bbde-cd9cc2a82917 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:48.358102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 206/410] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:48 2022] GET /identity/v3/projects/116e4cb137e24acda75f964e89e322c5 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:48.367470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782350db-37d0-49d0-bbde-cd9cc2a82917 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:48.398962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:48.419144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04c5d5-c206-47f8-91ae-868bf730dde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:48.419820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c5d5-c206-47f8-91ae-868bf730dde3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:48.420105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c5d5-c206-47f8-91ae-868bf730dde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:48.420385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04c5d5-c206-47f8-91ae-868bf730dde3 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:48.421133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04c5d5-c206-47f8-91ae-868bf730dde3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:48.421443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04c5d5-c206-47f8-91ae-868bf730dde3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:48.422858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04c5d5-c206-47f8-91ae-868bf730dde3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:48.443609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 205/411] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:48.449387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04c5d5-c206-47f8-91ae-868bf730dde3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:48.516750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782350db-37d0-49d0-bbde-cd9cc2a82917 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:48.556709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912e6d0-ec95-4e23-bff3-d3535bf3a941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:48.558110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912e6d0-ec95-4e23-bff3-d3535bf3a941 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:48.558402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912e6d0-ec95-4e23-bff3-d3535bf3a941 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:48.558712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912e6d0-ec95-4e23-bff3-d3535bf3a941 None admin] PATH_INFO: `/v3/projects/116e4cb137e24acda75f964e89e322c5/groups/1e126b8a6f9743459e30e085be179a4f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:48.561241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912e6d0-ec95-4e23-bff3-d3535bf3a941 None admin] RBAC: Authorizing `identity:create_grant(project_id=116e4cb137e24acda75f964e89e322c5, group_id=1e126b8a6f9743459e30e085be179a4f, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:48.562735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912e6d0-ec95-4e23-bff3-d3535bf3a941 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:48.580954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 207/412] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 26 20:50:48 2022] PUT /identity/v3/projects/116e4cb137e24acda75f964e89e322c5/groups/1e126b8a6f9743459e30e085be179a4f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:50:48.661173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04c5d5-c206-47f8-91ae-868bf730dde3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:48.721247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa17577-1c6e-42ec-a845-7fc91aa7cf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:48.722048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa17577-1c6e-42ec-a845-7fc91aa7cf60 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:48.722329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa17577-1c6e-42ec-a845-7fc91aa7cf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:48.722649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa17577-1c6e-42ec-a845-7fc91aa7cf60 None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:48.723897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa17577-1c6e-42ec-a845-7fc91aa7cf60 None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:48.725421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa17577-1c6e-42ec-a845-7fc91aa7cf60 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:48.731682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 206/413] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:48 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:48.733113 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912e6d0-ec95-4e23-bff3-d3535bf3a941 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:48.820445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:48.869217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912e6d0-ec95-4e23-bff3-d3535bf3a941 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:48.907901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c98f23-fb67-489d-85c2-be3f07b9919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:48.909017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c98f23-fb67-489d-85c2-be3f07b9919f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:48.909443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c98f23-fb67-489d-85c2-be3f07b9919f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:48.909860 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c98f23-fb67-489d-85c2-be3f07b9919f None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:48.911216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c98f23-fb67-489d-85c2-be3f07b9919f None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:48.912841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c98f23-fb67-489d-85c2-be3f07b9919f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:48.915012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 208/414] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:48 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:48.920582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa17577-1c6e-42ec-a845-7fc91aa7cf60 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:48.987463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:49.036824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa17577-1c6e-42ec-a845-7fc91aa7cf60 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:49.078347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005a541-dc1e-41ec-993a-594aa66edd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:49.079276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005a541-dc1e-41ec-993a-594aa66edd08 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:49.079559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005a541-dc1e-41ec-993a-594aa66edd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:49.079834 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005a541-dc1e-41ec-993a-594aa66edd08 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573/groups/8d5629ebc0bc41cab9b6560424962390/roles/fa67cb2f211d433097f606ff029f863e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:49.082036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005a541-dc1e-41ec-993a-594aa66edd08 None admin] RBAC: Authorizing `identity:create_grant(project_id=42dcbe790c2a48f2bd52298af1b58573, group_id=8d5629ebc0bc41cab9b6560424962390, role_id=fa67cb2f211d433097f606ff029f863e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:49.083504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005a541-dc1e-41ec-993a-594aa66edd08 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:49.112715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 207/415] 10.222.0.57 () {58 vars in 1442 bytes} [Thu May 26 20:50:48 2022] PUT /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573/groups/8d5629ebc0bc41cab9b6560424962390/roles/fa67cb2f211d433097f606ff029f863e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:50:49.331217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072146ea-09e5-4eac-a22c-40613d65af7d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:49.331868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072146ea-09e5-4eac-a22c-40613d65af7d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:49.332362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072146ea-09e5-4eac-a22c-40613d65af7d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:49.334259 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 209/416] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:49.340397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac6aa3-f06e-4a0d-86e8-36061b55a608 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:49.340981 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac6aa3-f06e-4a0d-86e8-36061b55a608 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:49.341495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac6aa3-f06e-4a0d-86e8-36061b55a608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:49.367581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d9ac6aa3-f06e-4a0d-86e8-36061b55a608 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:49.369756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d9ac6aa3-f06e-4a0d-86e8-36061b55a608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:49.459824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 208/417] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:49.496118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072146ea-09e5-4eac-a22c-40613d65af7d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:49.612686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072146ea-09e5-4eac-a22c-40613d65af7d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:49.640528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:49.656892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b157e3-7fca-4afc-a9bd-0add00552a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:49.657751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b157e3-7fca-4afc-a9bd-0add00552a2c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:49.658036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b157e3-7fca-4afc-a9bd-0add00552a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:49.658309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b157e3-7fca-4afc-a9bd-0add00552a2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:49.674264 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b157e3-7fca-4afc-a9bd-0add00552a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b7b157e3-7fca-4afc-a9bd-0add00552a2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:49.683115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:49.684014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:49.684838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:49.688224 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 210/418] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:49.691891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac6aa3-f06e-4a0d-86e8-36061b55a608 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:49.777203 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:49.853322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac6aa3-f06e-4a0d-86e8-36061b55a608 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:49.905115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548a69d-e253-4a5e-a9c4-6d5754ae855a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:49.905720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548a69d-e253-4a5e-a9c4-6d5754ae855a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:49.906000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548a69d-e253-4a5e-a9c4-6d5754ae855a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:49.906278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548a69d-e253-4a5e-a9c4-6d5754ae855a None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:49.907127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548a69d-e253-4a5e-a9c4-6d5754ae855a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:49.907438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548a69d-e253-4a5e-a9c4-6d5754ae855a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:49.908812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548a69d-e253-4a5e-a9c4-6d5754ae855a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:49.922547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 209/419] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:49.931977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b157e3-7fca-4afc-a9bd-0add00552a2c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:50.074196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b157e3-7fca-4afc-a9bd-0add00552a2c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:50.121926 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64a025-f6f6-4f3f-a2d9-7de51504bfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:50.122605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64a025-f6f6-4f3f-a2d9-7de51504bfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:50.122891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64a025-f6f6-4f3f-a2d9-7de51504bfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:50.123167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64a025-f6f6-4f3f-a2d9-7de51504bfc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:50.169576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b15a9-1b95-4ce2-9a96-2d0f1f003c53 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:50.169576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b15a9-1b95-4ce2-9a96-2d0f1f003c53 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:50.169576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b15a9-1b95-4ce2-9a96-2d0f1f003c53 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:50.174143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 210/420] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:50:50.176140 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28d61c-b4eb-40a2-b6db-02a2efe6e330 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:50.176436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28d61c-b4eb-40a2-b6db-02a2efe6e330 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:50.176709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28d61c-b4eb-40a2-b6db-02a2efe6e330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:50.182812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:50.203848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4d28d61c-b4eb-40a2-b6db-02a2efe6e330 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:50.206703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4d28d61c-b4eb-40a2-b6db-02a2efe6e330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:50.222482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64a025-f6f6-4f3f-a2d9-7de51504bfc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:50.223951 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64a025-f6f6-4f3f-a2d9-7de51504bfc5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:50.306993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 211/421] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:50.330855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28d61c-b4eb-40a2-b6db-02a2efe6e330 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:50.402817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:50.442023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 211/422] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:50.443426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af23559-3f67-4854-90af-bbb4c973943c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:50.443769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af23559-3f67-4854-90af-bbb4c973943c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:50.444038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af23559-3f67-4854-90af-bbb4c973943c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:50.446571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 212/423] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:50.451448 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af23559-3f67-4854-90af-bbb4c973943c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:50.479349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28d61c-b4eb-40a2-b6db-02a2efe6e330 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:50.522956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0608cda-c32b-4afe-8c8a-a04222320d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:50.525200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0608cda-c32b-4afe-8c8a-a04222320d90 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:50.525957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0608cda-c32b-4afe-8c8a-a04222320d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:50.526438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0608cda-c32b-4afe-8c8a-a04222320d90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:50.543364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0608cda-c32b-4afe-8c8a-a04222320d90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:50.545571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e0608cda-c32b-4afe-8c8a-a04222320d90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:50.552681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:50.553926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:50.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:50.562883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 212/424] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:50.563836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c4cb0-4f43-413e-9ecd-c0b49e28b211 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:50.564103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c4cb0-4f43-413e-9ecd-c0b49e28b211 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:50.564387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c4cb0-4f43-413e-9ecd-c0b49e28b211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:50.567962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af23559-3f67-4854-90af-bbb4c973943c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:50.586151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-701c4cb0-4f43-413e-9ecd-c0b49e28b211 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:50.588054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-701c4cb0-4f43-413e-9ecd-c0b49e28b211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:50.604331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa0aa9-a5c5-4f61-9528-6f3f38d581c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:50.604984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa0aa9-a5c5-4f61-9528-6f3f38d581c6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:50.605270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa0aa9-a5c5-4f61-9528-6f3f38d581c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:50.605826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa0aa9-a5c5-4f61-9528-6f3f38d581c6 None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:50.606959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa0aa9-a5c5-4f61-9528-6f3f38d581c6 None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:50.608407 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa0aa9-a5c5-4f61-9528-6f3f38d581c6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:50.614666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 213/425] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:50 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:50.615488 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa0aa9-a5c5-4f61-9528-6f3f38d581c6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:50.686510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 213/426] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:50.687611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c4cb0-4f43-413e-9ecd-c0b49e28b211 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:50.725052 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa0aa9-a5c5-4f61-9528-6f3f38d581c6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:50.733923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:50.756286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a1300-2de4-4184-a121-ffd2be469f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:50.756959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a1300-2de4-4184-a121-ffd2be469f1b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:50.757245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a1300-2de4-4184-a121-ffd2be469f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:50.757562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a1300-2de4-4184-a121-ffd2be469f1b None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:50.758267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a1300-2de4-4184-a121-ffd2be469f1b None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:50.758614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a1300-2de4-4184-a121-ffd2be469f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:50.760533 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a1300-2de4-4184-a121-ffd2be469f1b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:50.773934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 214/427] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:50.773934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a1300-2de4-4184-a121-ffd2be469f1b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:50.798609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:50.835419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c4cb0-4f43-413e-9ecd-c0b49e28b211 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:50.882031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317649fb-9d58-46b9-bf61-97fc286d3631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:50.883431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317649fb-9d58-46b9-bf61-97fc286d3631 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:50.884616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317649fb-9d58-46b9-bf61-97fc286d3631 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:50.885038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317649fb-9d58-46b9-bf61-97fc286d3631 None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:50.887309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317649fb-9d58-46b9-bf61-97fc286d3631 None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:50.889575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317649fb-9d58-46b9-bf61-97fc286d3631 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:50.891915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 214/428] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:50 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:50.893578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317649fb-9d58-46b9-bf61-97fc286d3631 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:50.895453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a1300-2de4-4184-a121-ffd2be469f1b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:50.939481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d7995-a50c-4f4a-8e1f-d48999cfe128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:50.941606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d7995-a50c-4f4a-8e1f-d48999cfe128 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:50.942037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d7995-a50c-4f4a-8e1f-d48999cfe128 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:50.942508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d7995-a50c-4f4a-8e1f-d48999cfe128 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:50.957338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8d7995-a50c-4f4a-8e1f-d48999cfe128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ed8d7995-a50c-4f4a-8e1f-d48999cfe128 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:50.965649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:50.966601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 20:50:50.967419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:50.970752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 215/429] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:50.971618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d7995-a50c-4f4a-8e1f-d48999cfe128 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:51.003179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:51.009844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317649fb-9d58-46b9-bf61-97fc286d3631 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:51.046951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96ba5d-a4e4-404e-bc8f-766b99d85137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:51.047823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96ba5d-a4e4-404e-bc8f-766b99d85137 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.048211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96ba5d-a4e4-404e-bc8f-766b99d85137 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.048604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef96ba5d-a4e4-404e-bc8f-766b99d85137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.077314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:51.108600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d7995-a50c-4f4a-8e1f-d48999cfe128 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:51.123771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96ba5d-a4e4-404e-bc8f-766b99d85137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:51.125465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef96ba5d-a4e4-404e-bc8f-766b99d85137 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:51.151106 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be63a5-4ffb-4e84-b460-776b1ea5f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:51.152448 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be63a5-4ffb-4e84-b460-776b1ea5f064 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.152879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be63a5-4ffb-4e84-b460-776b1ea5f064 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.153559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be63a5-4ffb-4e84-b460-776b1ea5f064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.154554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be63a5-4ffb-4e84-b460-776b1ea5f064 None admin] RBAC: Adding query filter params (group.id=8d5629ebc0bc41cab9b6560424962390, role.id=fa67cb2f211d433097f606ff029f863e, scope.project.id=bb614e1d464d4df1959debd82ba5049c) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:51.155155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be63a5-4ffb-4e84-b460-776b1ea5f064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:51.157370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be63a5-4ffb-4e84-b460-776b1ea5f064 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:51.165652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 216/430] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 26 20:50:50 2022] GET /identity/v3/role_assignments?group.id=8d5629ebc0bc41cab9b6560424962390&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=fa67cb2f211d433097f606ff029f863e => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:51.166726 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be63a5-4ffb-4e84-b460-776b1ea5f064 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:51.236609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:51.249605 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 215/431] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:51.261905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96ba5d-a4e4-404e-bc8f-766b99d85137 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:51.309766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be63a5-4ffb-4e84-b460-776b1ea5f064 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:51.350334 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d9782-7446-4002-8b8f-a264a8777b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:51.351449 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d9782-7446-4002-8b8f-a264a8777b17 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.351909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d9782-7446-4002-8b8f-a264a8777b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.352324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d9782-7446-4002-8b8f-a264a8777b17 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.353556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d9782-7446-4002-8b8f-a264a8777b17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:51.354009 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d9782-7446-4002-8b8f-a264a8777b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:51.356135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d9782-7446-4002-8b8f-a264a8777b17 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:51.362499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:51.369495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 217/432] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:50:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:51.379594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d9782-7446-4002-8b8f-a264a8777b17 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:51.440402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef96ba5d-a4e4-404e-bc8f-766b99d85137 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:51.481789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5fe54b-aff0-4007-9520-63268c23250e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:51.482769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe54b-aff0-4007-9520-63268c23250e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.483016 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe54b-aff0-4007-9520-63268c23250e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.483320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5fe54b-aff0-4007-9520-63268c23250e None admin] PATH_INFO: `/v3/projects/bb614e1d464d4df1959debd82ba5049c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.485265 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fe54b-aff0-4007-9520-63268c23250e None admin] RBAC: Authorizing `identity:get_project(project_id=bb614e1d464d4df1959debd82ba5049c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:51.486749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5fe54b-aff0-4007-9520-63268c23250e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:51.488779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 216/433] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:51 2022] GET /identity/v3/projects/bb614e1d464d4df1959debd82ba5049c => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:51.490170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccffc62-1c13-47c9-982a-661f1eb442b0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.490551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccffc62-1c13-47c9-982a-661f1eb442b0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.490840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccffc62-1c13-47c9-982a-661f1eb442b0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.492330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 217/434] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:51.497817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccffc62-1c13-47c9-982a-661f1eb442b0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:51.526596 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d9782-7446-4002-8b8f-a264a8777b17 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:51.573352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3c613-1cce-4069-b690-14ab9f7e650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:51.574002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3c613-1cce-4069-b690-14ab9f7e650e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.574287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3c613-1cce-4069-b690-14ab9f7e650e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.574591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3c613-1cce-4069-b690-14ab9f7e650e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.584796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:51.623884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccffc62-1c13-47c9-982a-661f1eb442b0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:51.653435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:51.655986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3c613-1cce-4069-b690-14ab9f7e650e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:51.657885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3c613-1cce-4069-b690-14ab9f7e650e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:51.663987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5df92-aff4-4f5a-92c2-67dad850c562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:51.664840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5df92-aff4-4f5a-92c2-67dad850c562 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.665643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5df92-aff4-4f5a-92c2-67dad850c562 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.665919 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5df92-aff4-4f5a-92c2-67dad850c562 None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.667555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5df92-aff4-4f5a-92c2-67dad850c562 None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:51.669347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5df92-aff4-4f5a-92c2-67dad850c562 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:51.671334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 218/435] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:51 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:51.671334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df9552-c341-4de1-a4f5-5009cdca3a78 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.671334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df9552-c341-4de1-a4f5-5009cdca3a78 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.671334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df9552-c341-4de1-a4f5-5009cdca3a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.692489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-57df9552-c341-4de1-a4f5-5009cdca3a78 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:51.693679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-57df9552-c341-4de1-a4f5-5009cdca3a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:51.770503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 219/436] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:51.770503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df9552-c341-4de1-a4f5-5009cdca3a78 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:51.805144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 218/437] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:51.806790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3c613-1cce-4069-b690-14ab9f7e650e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:51.850718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:51.893217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df9552-c341-4de1-a4f5-5009cdca3a78 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:51.914654 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:51.918782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3c613-1cce-4069-b690-14ab9f7e650e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:51.929707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd29c8d-a78a-4cd5-9acb-31ab0ee7e9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:51.930373 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd29c8d-a78a-4cd5-9acb-31ab0ee7e9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.930716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd29c8d-a78a-4cd5-9acb-31ab0ee7e9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.931012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd29c8d-a78a-4cd5-9acb-31ab0ee7e9e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.931833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd29c8d-a78a-4cd5-9acb-31ab0ee7e9e5 None admin] RBAC: Adding query filter params (group.id=8d5629ebc0bc41cab9b6560424962390, role.id=d3bdf57150744faf9c5c25c317df48fc, scope.project.id=bb614e1d464d4df1959debd82ba5049c) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:51.932142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd29c8d-a78a-4cd5-9acb-31ab0ee7e9e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:51.933569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd29c8d-a78a-4cd5-9acb-31ab0ee7e9e5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:51.939236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 220/438] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 26 20:50:51 2022] GET /identity/v3/role_assignments?group.id=8d5629ebc0bc41cab9b6560424962390&scope.project.id=bb614e1d464d4df1959debd82ba5049c&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:51.939998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd29c8d-a78a-4cd5-9acb-31ab0ee7e9e5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:51.953474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87639e5-1803-4a7a-9412-6b06b57c32d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:51.954291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87639e5-1803-4a7a-9412-6b06b57c32d5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.954619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87639e5-1803-4a7a-9412-6b06b57c32d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.954901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87639e5-1803-4a7a-9412-6b06b57c32d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.967047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87639e5-1803-4a7a-9412-6b06b57c32d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d87639e5-1803-4a7a-9412-6b06b57c32d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:51.975060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:51.976038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 219/439] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:50:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:51.987496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6a02c-9eb1-46a9-8c1f-825d3ed547a1 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:51.988237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6a02c-9eb1-46a9-8c1f-825d3ed547a1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:51.988237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6a02c-9eb1-46a9-8c1f-825d3ed547a1 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:51.988237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 220/440] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:51.988237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6a02c-9eb1-46a9-8c1f-825d3ed547a1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:52.092497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd29c8d-a78a-4cd5-9acb-31ab0ee7e9e5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:52.117963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6a02c-9eb1-46a9-8c1f-825d3ed547a1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:52.119940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:52.134797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bafd3-d24a-41e7-ad22-9d7bd2e1eaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:52.135811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bafd3-d24a-41e7-ad22-9d7bd2e1eaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:52.136100 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bafd3-d24a-41e7-ad22-9d7bd2e1eaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:52.137238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bafd3-d24a-41e7-ad22-9d7bd2e1eaa6 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:52.137838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bafd3-d24a-41e7-ad22-9d7bd2e1eaa6 None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:52.139696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bafd3-d24a-41e7-ad22-9d7bd2e1eaa6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:52.141177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 221/441] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:51 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:52.142140 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927e37d-2c3a-4a64-a393-844094a7ea8b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:52.142428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927e37d-2c3a-4a64-a393-844094a7ea8b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:52.142734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927e37d-2c3a-4a64-a393-844094a7ea8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:52.160370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f927e37d-2c3a-4a64-a393-844094a7ea8b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:52.161559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f927e37d-2c3a-4a64-a393-844094a7ea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:52.162627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571e21-62f0-44bb-919b-a875ee92206b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:52.163273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571e21-62f0-44bb-919b-a875ee92206b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:52.163559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571e21-62f0-44bb-919b-a875ee92206b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:52.163830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6571e21-62f0-44bb-919b-a875ee92206b None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:52.166060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571e21-62f0-44bb-919b-a875ee92206b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:52.166060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571e21-62f0-44bb-919b-a875ee92206b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:52.166930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6571e21-62f0-44bb-919b-a875ee92206b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:52.176208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 221/442] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:50:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:52.178845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571e21-62f0-44bb-919b-a875ee92206b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:52.207975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:52.243321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 222/443] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:52.244194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927e37d-2c3a-4a64-a393-844094a7ea8b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:52.297794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6571e21-62f0-44bb-919b-a875ee92206b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:52.346282 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e7a96-32fb-42f8-bf55-31bec9d73d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:52.347547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e7a96-32fb-42f8-bf55-31bec9d73d83 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:52.347833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e7a96-32fb-42f8-bf55-31bec9d73d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:52.348105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e7a96-32fb-42f8-bf55-31bec9d73d83 None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:52.349419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e7a96-32fb-42f8-bf55-31bec9d73d83 None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:52.350922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e7a96-32fb-42f8-bf55-31bec9d73d83 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:52.353200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 222/444] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:52 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:52.353578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e7a96-32fb-42f8-bf55-31bec9d73d83 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:52.379413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927e37d-2c3a-4a64-a393-844094a7ea8b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:52.405498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:52.418151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaeeea-161f-4e3c-8a5c-79b9dea2c5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:52.418857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaeeea-161f-4e3c-8a5c-79b9dea2c5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:52.419141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaeeea-161f-4e3c-8a5c-79b9dea2c5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:52.419415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaeeea-161f-4e3c-8a5c-79b9dea2c5f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:52.465197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:52.477011 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e7a96-32fb-42f8-bf55-31bec9d73d83 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:52.500353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaeeea-161f-4e3c-8a5c-79b9dea2c5f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:52.501940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaeeea-161f-4e3c-8a5c-79b9dea2c5f2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:52.518053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f13fc-8606-4832-8876-c4cf83b7e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:52.521635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f13fc-8606-4832-8876-c4cf83b7e068 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:52.521635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f13fc-8606-4832-8876-c4cf83b7e068 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:52.521635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f13fc-8606-4832-8876-c4cf83b7e068 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:52.529776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f13fc-8606-4832-8876-c4cf83b7e068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-320f13fc-8606-4832-8876-c4cf83b7e068 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:52.538025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:52.538934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:52.539755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:52.544674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 223/445] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:52.545398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f13fc-8606-4832-8876-c4cf83b7e068 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:52.605990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 223/446] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:50:52.606776 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaeeea-161f-4e3c-8a5c-79b9dea2c5f2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:52.635221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:52.644057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f13fc-8606-4832-8876-c4cf83b7e068 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:52.676963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647024bf-c3e4-4548-9889-ff5e44c9128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:52.677612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647024bf-c3e4-4548-9889-ff5e44c9128f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:52.677903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647024bf-c3e4-4548-9889-ff5e44c9128f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:52.678186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647024bf-c3e4-4548-9889-ff5e44c9128f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:52.679054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647024bf-c3e4-4548-9889-ff5e44c9128f None admin] RBAC: Adding query filter params (group.id=8d5629ebc0bc41cab9b6560424962390, role.id=d3bdf57150744faf9c5c25c317df48fc, scope.project.id=42dcbe790c2a48f2bd52298af1b58573) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:52.679411 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647024bf-c3e4-4548-9889-ff5e44c9128f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:52.680970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647024bf-c3e4-4548-9889-ff5e44c9128f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:52.686823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 224/447] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 26 20:50:52 2022] GET /identity/v3/role_assignments?group.id=8d5629ebc0bc41cab9b6560424962390&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=d3bdf57150744faf9c5c25c317df48fc => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:52.687612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647024bf-c3e4-4548-9889-ff5e44c9128f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:52.712895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:52.720783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaeeea-161f-4e3c-8a5c-79b9dea2c5f2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:52.761094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2243f3-b2ce-4fbc-a949-1933c6ef7740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:52.762329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2243f3-b2ce-4fbc-a949-1933c6ef7740 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:52.762656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2243f3-b2ce-4fbc-a949-1933c6ef7740 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:52.762918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2243f3-b2ce-4fbc-a949-1933c6ef7740 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:52.763676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2243f3-b2ce-4fbc-a949-1933c6ef7740 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:52.764053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2243f3-b2ce-4fbc-a949-1933c6ef7740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:52.765514 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2243f3-b2ce-4fbc-a949-1933c6ef7740 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:52.775562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 224/448] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:52.781300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2243f3-b2ce-4fbc-a949-1933c6ef7740 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:52.818611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647024bf-c3e4-4548-9889-ff5e44c9128f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:52.856275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf24678-c207-4f1a-acab-0849ac0442c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:52.857159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf24678-c207-4f1a-acab-0849ac0442c1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:52.857452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf24678-c207-4f1a-acab-0849ac0442c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:52.857733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf24678-c207-4f1a-acab-0849ac0442c1 None admin] PATH_INFO: `/v3/projects/116e4cb137e24acda75f964e89e322c5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:52.858995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf24678-c207-4f1a-acab-0849ac0442c1 None admin] RBAC: Authorizing `identity:get_project(project_id=116e4cb137e24acda75f964e89e322c5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:52.860465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf24678-c207-4f1a-acab-0849ac0442c1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:52.862188 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 225/449] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:52 2022] GET /identity/v3/projects/116e4cb137e24acda75f964e89e322c5 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:52.867647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf24678-c207-4f1a-acab-0849ac0442c1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:52.885342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:52.903511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2243f3-b2ce-4fbc-a949-1933c6ef7740 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:52.941683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665c4f-a72a-48ce-a404-491a3827f40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:52.942348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665c4f-a72a-48ce-a404-491a3827f40b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:52.942666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665c4f-a72a-48ce-a404-491a3827f40b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:52.943703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665c4f-a72a-48ce-a404-491a3827f40b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:52.965000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:52.989635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf24678-c207-4f1a-acab-0849ac0442c1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:53.028678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71665c4f-a72a-48ce-a404-491a3827f40b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:53.030139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71665c4f-a72a-48ce-a404-491a3827f40b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:53.032507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e364c9-42dc-4eea-87e8-2587cfb1afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:53.033425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e364c9-42dc-4eea-87e8-2587cfb1afb9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:53.033714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e364c9-42dc-4eea-87e8-2587cfb1afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:53.033995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e364c9-42dc-4eea-87e8-2587cfb1afb9 None admin] PATH_INFO: `/v3/groups/1e126b8a6f9743459e30e085be179a4f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:53.035215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e364c9-42dc-4eea-87e8-2587cfb1afb9 None admin] RBAC: Authorizing `identity:get_group(group_id=1e126b8a6f9743459e30e085be179a4f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:53.036649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e364c9-42dc-4eea-87e8-2587cfb1afb9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:53.038557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 226/450] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:52 2022] GET /identity/v3/groups/1e126b8a6f9743459e30e085be179a4f => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:53.044158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e364c9-42dc-4eea-87e8-2587cfb1afb9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:53.118724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:53.135074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 225/451] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:50:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:50:53.140932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665c4f-a72a-48ce-a404-491a3827f40b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:53.148787 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e364c9-42dc-4eea-87e8-2587cfb1afb9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:53.181838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279e0e1-518b-4767-81a6-c1fb06f19af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:53.182518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279e0e1-518b-4767-81a6-c1fb06f19af9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:53.182810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279e0e1-518b-4767-81a6-c1fb06f19af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:53.183092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279e0e1-518b-4767-81a6-c1fb06f19af9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:53.183925 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279e0e1-518b-4767-81a6-c1fb06f19af9 None admin] RBAC: Adding query filter params (group.id=1e126b8a6f9743459e30e085be179a4f, role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=116e4cb137e24acda75f964e89e322c5) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:53.184237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279e0e1-518b-4767-81a6-c1fb06f19af9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:53.185692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279e0e1-518b-4767-81a6-c1fb06f19af9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:53.191420 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 227/452] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 26 20:50:53 2022] GET /identity/v3/role_assignments?group.id=1e126b8a6f9743459e30e085be179a4f&scope.project.id=116e4cb137e24acda75f964e89e322c5&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:53.239035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665c4f-a72a-48ce-a404-491a3827f40b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:53.275644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288254a1-5652-4802-8261-45e7d0cbae66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:53.276522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288254a1-5652-4802-8261-45e7d0cbae66 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:53.276805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288254a1-5652-4802-8261-45e7d0cbae66 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:53.277081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288254a1-5652-4802-8261-45e7d0cbae66 None admin] PATH_INFO: `/v3/projects/42dcbe790c2a48f2bd52298af1b58573` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:53.278298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288254a1-5652-4802-8261-45e7d0cbae66 None admin] RBAC: Authorizing `identity:get_project(project_id=42dcbe790c2a48f2bd52298af1b58573)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:53.279805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288254a1-5652-4802-8261-45e7d0cbae66 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:53.281526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 226/453] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:50:53 2022] GET /identity/v3/projects/42dcbe790c2a48f2bd52298af1b58573 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:50:53.287312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279e0e1-518b-4767-81a6-c1fb06f19af9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:53.303925 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:53.391331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279e0e1-518b-4767-81a6-c1fb06f19af9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:53.423642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957335ec-c528-4315-bab8-0ed4091d3218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:53.424488 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957335ec-c528-4315-bab8-0ed4091d3218 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:53.424771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957335ec-c528-4315-bab8-0ed4091d3218 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:53.425052 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957335ec-c528-4315-bab8-0ed4091d3218 None admin] PATH_INFO: `/v3/groups/8d5629ebc0bc41cab9b6560424962390` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:53.426191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957335ec-c528-4315-bab8-0ed4091d3218 None admin] RBAC: Authorizing `identity:get_group(group_id=8d5629ebc0bc41cab9b6560424962390)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:53.427694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957335ec-c528-4315-bab8-0ed4091d3218 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:53.429288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 228/454] 10.222.0.57 () {56 vars in 1180 bytes} [Thu May 26 20:50:53 2022] GET /identity/v3/groups/8d5629ebc0bc41cab9b6560424962390 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:53.435731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288254a1-5652-4802-8261-45e7d0cbae66 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:53.488342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:53.531647 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288254a1-5652-4802-8261-45e7d0cbae66 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:53.564652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b981b1a0-b4ec-4932-9fa2-ef4706844466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:53.565283 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981b1a0-b4ec-4932-9fa2-ef4706844466 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:53.565566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981b1a0-b4ec-4932-9fa2-ef4706844466 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:53.565841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b981b1a0-b4ec-4932-9fa2-ef4706844466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:53.566676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981b1a0-b4ec-4932-9fa2-ef4706844466 None admin] RBAC: Adding query filter params (group.id=8d5629ebc0bc41cab9b6560424962390, role.id=fa67cb2f211d433097f606ff029f863e, scope.project.id=42dcbe790c2a48f2bd52298af1b58573) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:53.566986 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981b1a0-b4ec-4932-9fa2-ef4706844466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:53.568397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b981b1a0-b4ec-4932-9fa2-ef4706844466 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:53.574605 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 227/455] 10.222.0.57 () {56 vars in 1376 bytes} [Thu May 26 20:50:53 2022] GET /identity/v3/role_assignments?group.id=8d5629ebc0bc41cab9b6560424962390&scope.project.id=42dcbe790c2a48f2bd52298af1b58573&role.id=fa67cb2f211d433097f606ff029f863e => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:56.474395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8edfab-d90c-4e8e-aacb-d9d3aa0f32ab None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:56.475286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8edfab-d90c-4e8e-aacb-d9d3aa0f32ab None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:56.475762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8edfab-d90c-4e8e-aacb-d9d3aa0f32ab None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:56.477275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 229/456] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:56.482733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3f046-0262-4d1d-a3fb-9f2d160c2718 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:56.483527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3f046-0262-4d1d-a3fb-9f2d160c2718 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:56.483957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3f046-0262-4d1d-a3fb-9f2d160c2718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:56.511972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-44a3f046-0262-4d1d-a3fb-9f2d160c2718 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:56.513455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-44a3f046-0262-4d1d-a3fb-9f2d160c2718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:56.531673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd276c-32b1-4022-b0d2-cbd840fd8dd7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:56.532294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd276c-32b1-4022-b0d2-cbd840fd8dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:56.532771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd276c-32b1-4022-b0d2-cbd840fd8dd7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:56.534717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 230/457] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:56.540841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd96b5-5dfd-4f10-a1f6-56f42c86e295 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:56.541458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd96b5-5dfd-4f10-a1f6-56f42c86e295 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:56.541928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd96b5-5dfd-4f10-a1f6-56f42c86e295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:56.572153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9dbd96b5-5dfd-4f10-a1f6-56f42c86e295 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:56.573696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9dbd96b5-5dfd-4f10-a1f6-56f42c86e295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:56.628055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 228/458] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:56.655730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3f046-0262-4d1d-a3fb-9f2d160c2718 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:56.672143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 231/459] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:50:56.699129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd96b5-5dfd-4f10-a1f6-56f42c86e295 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:56.714324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:56.770891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:56.819013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3f046-0262-4d1d-a3fb-9f2d160c2718 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:56.822954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd96b5-5dfd-4f10-a1f6-56f42c86e295 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:56.858828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989c3f0-4b52-4715-ba00-73e28e85f820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:56.859905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989c3f0-4b52-4715-ba00-73e28e85f820 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:56.860438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989c3f0-4b52-4715-ba00-73e28e85f820 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:56.860933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989c3f0-4b52-4715-ba00-73e28e85f820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:56.865719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6af72-6372-4365-a553-7a144ae0ea0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:56.866556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6af72-6372-4365-a553-7a144ae0ea0f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:56.866840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6af72-6372-4365-a553-7a144ae0ea0f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:56.867115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6af72-6372-4365-a553-7a144ae0ea0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:56.874572 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989c3f0-4b52-4715-ba00-73e28e85f820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:56.876312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3989c3f0-4b52-4715-ba00-73e28e85f820 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:56.879293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6af72-6372-4365-a553-7a144ae0ea0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:56.880917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6af72-6372-4365-a553-7a144ae0ea0f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3989c3f0-4b52-4715-ba00-73e28e85f820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:50:56.895221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:56.896286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:56.897114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:50:56.897114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:50:56.897114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:50:56.897114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:56.897335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 229/460] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:56.902031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dff581-51c7-42f3-92eb-39c7bf867e3b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:56.902031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dff581-51c7-42f3-92eb-39c7bf867e3b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:56.902031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dff581-51c7-42f3-92eb-39c7bf867e3b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:56.902031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 230/461] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:56.906322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b619e-4c67-4017-b636-56bda88f8508 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:56.906709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b619e-4c67-4017-b636-56bda88f8508 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:56.907068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b619e-4c67-4017-b636-56bda88f8508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d0b6af72-6372-4365-a553-7a144ae0ea0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:50:56.908394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:56.909209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:56.910087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:50:56.910087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:50:56.910087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:50:56.910087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:56.912279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 232/462] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:56.913295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6af72-6372-4365-a553-7a144ae0ea0f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:56.927330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-205b619e-4c67-4017-b636-56bda88f8508 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:56.928645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-205b619e-4c67-4017-b636-56bda88f8508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:56.981344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:57.006576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 231/463] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:57.007736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b619e-4c67-4017-b636-56bda88f8508 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:57.023173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6af72-6372-4365-a553-7a144ae0ea0f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:57.053179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:57.060086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e210a2-77bc-40ba-829a-d3ff4ad9dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:57.060760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e210a2-77bc-40ba-829a-d3ff4ad9dfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.061205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e210a2-77bc-40ba-829a-d3ff4ad9dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.061598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e210a2-77bc-40ba-829a-d3ff4ad9dfb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.062332 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e210a2-77bc-40ba-829a-d3ff4ad9dfb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:57.062684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e210a2-77bc-40ba-829a-d3ff4ad9dfb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:57.064082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e210a2-77bc-40ba-829a-d3ff4ad9dfb7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:57.088600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 233/464] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:57.088600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf92846-03fc-4ad9-9488-88724068fd2a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.088600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf92846-03fc-4ad9-9488-88724068fd2a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.088600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf92846-03fc-4ad9-9488-88724068fd2a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.088600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 234/465] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:57.088600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf92846-03fc-4ad9-9488-88724068fd2a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:57.135175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b619e-4c67-4017-b636-56bda88f8508 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:57.172150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06488e-b5b5-48ff-a215-74ba2d8c2d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:57.172999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06488e-b5b5-48ff-a215-74ba2d8c2d8f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.173329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06488e-b5b5-48ff-a215-74ba2d8c2d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.173615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06488e-b5b5-48ff-a215-74ba2d8c2d8f None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.174339 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06488e-b5b5-48ff-a215-74ba2d8c2d8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:57.174680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06488e-b5b5-48ff-a215-74ba2d8c2d8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:57.176311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06488e-b5b5-48ff-a215-74ba2d8c2d8f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:57.189014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 232/466] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:57.189877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06488e-b5b5-48ff-a215-74ba2d8c2d8f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:57.200088 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf92846-03fc-4ad9-9488-88724068fd2a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:57.224969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:57.238406 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfe8a4-49bc-4ac4-b89e-a3a2e4dbc1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:57.239228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe8a4-49bc-4ac4-b89e-a3a2e4dbc1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.239515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe8a4-49bc-4ac4-b89e-a3a2e4dbc1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.239791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe8a4-49bc-4ac4-b89e-a3a2e4dbc1d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.250542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfe8a4-49bc-4ac4-b89e-a3a2e4dbc1d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:57.252029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfe8a4-49bc-4ac4-b89e-a3a2e4dbc1d3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0ddfe8a4-49bc-4ac4-b89e-a3a2e4dbc1d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:50:57.269170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:57.270060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:57.271033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:50:57.271033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:50:57.271033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:50:57.271033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:57.275816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 235/467] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:57.277084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc8d47-3899-4a07-bd4a-6b9d83cf0236 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.277376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc8d47-3899-4a07-bd4a-6b9d83cf0236 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.277656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc8d47-3899-4a07-bd4a-6b9d83cf0236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.290782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:57.295580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b5cc8d47-3899-4a07-bd4a-6b9d83cf0236 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:57.296775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b5cc8d47-3899-4a07-bd4a-6b9d83cf0236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:57.302195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06488e-b5b5-48ff-a215-74ba2d8c2d8f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:57.337605 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804b5f8-a7ad-42c5-8fac-7cab3b8b29d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:57.338285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804b5f8-a7ad-42c5-8fac-7cab3b8b29d2 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.338604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804b5f8-a7ad-42c5-8fac-7cab3b8b29d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.338889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804b5f8-a7ad-42c5-8fac-7cab3b8b29d2 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.339596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804b5f8-a7ad-42c5-8fac-7cab3b8b29d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:57.341042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804b5f8-a7ad-42c5-8fac-7cab3b8b29d2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:57.370160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 233/468] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:50:57 2022] POST /identity/v3/users => generated 246 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:57.370160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804b5f8-a7ad-42c5-8fac-7cab3b8b29d2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:57.386992 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 236/469] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:57.387998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc8d47-3899-4a07-bd4a-6b9d83cf0236 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:57.508358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804b5f8-a7ad-42c5-8fac-7cab3b8b29d2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:57.533344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc8d47-3899-4a07-bd4a-6b9d83cf0236 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:57.542810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:57.559821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a42b45-ce06-423e-a8ee-839386e16ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:57.560512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a42b45-ce06-423e-a8ee-839386e16ddf None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.560796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a42b45-ce06-423e-a8ee-839386e16ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.561076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a42b45-ce06-423e-a8ee-839386e16ddf None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.561784 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a42b45-ce06-423e-a8ee-839386e16ddf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:57.563303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a42b45-ce06-423e-a8ee-839386e16ddf None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:57.577566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1043bc4-8602-4448-94a5-dfcd16172d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:57.578257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1043bc4-8602-4448-94a5-dfcd16172d25 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.578670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1043bc4-8602-4448-94a5-dfcd16172d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.578959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1043bc4-8602-4448-94a5-dfcd16172d25 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.579792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1043bc4-8602-4448-94a5-dfcd16172d25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:57.580153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1043bc4-8602-4448-94a5-dfcd16172d25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:57.581802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1043bc4-8602-4448-94a5-dfcd16172d25 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:57.594901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 234/470] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:50:57 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:57.596156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a42b45-ce06-423e-a8ee-839386e16ddf None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:57.598067 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 237/471] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:57.604893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1043bc4-8602-4448-94a5-dfcd16172d25 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:57.607216 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:57.731496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a42b45-ce06-423e-a8ee-839386e16ddf None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:57.769082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1043bc4-8602-4448-94a5-dfcd16172d25 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:57.769543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe05d5-441e-4df4-91f7-76fb709092cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:57.770570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe05d5-441e-4df4-91f7-76fb709092cd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.770862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe05d5-441e-4df4-91f7-76fb709092cd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.771146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe05d5-441e-4df4-91f7-76fb709092cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.787238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe05d5-441e-4df4-91f7-76fb709092cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:57.788737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe05d5-441e-4df4-91f7-76fb709092cd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b1fe05d5-441e-4df4-91f7-76fb709092cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:50:57.811997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:57.813056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:57.814142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:50:57.814142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:50:57.814142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:50:57.814142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:50:57.816553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 235/472] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:57.822556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c250-1ce4-4327-ae45-c1aae79d2d80 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.822556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c250-1ce4-4327-ae45-c1aae79d2d80 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.822556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c250-1ce4-4327-ae45-c1aae79d2d80 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.822556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 236/473] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:50:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:50:57.827885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3154b6b-f439-46f7-937d-59599be03295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:57.828584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3154b6b-f439-46f7-937d-59599be03295 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.828862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3154b6b-f439-46f7-937d-59599be03295 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.829142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3154b6b-f439-46f7-937d-59599be03295 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.829859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3154b6b-f439-46f7-937d-59599be03295 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:57.831367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3154b6b-f439-46f7-937d-59599be03295 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:57.831634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5c250-1ce4-4327-ae45-c1aae79d2d80 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:57.857866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 238/474] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:50:57 2022] POST /identity/v3/users => generated 248 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:57.859329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d98f6e-64b0-4f62-9237-2a8d7da82068 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:57.859716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d98f6e-64b0-4f62-9237-2a8d7da82068 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:57.860001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d98f6e-64b0-4f62-9237-2a8d7da82068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:57.878410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c0d98f6e-64b0-4f62-9237-2a8d7da82068 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:50:57.880052 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c0d98f6e-64b0-4f62-9237-2a8d7da82068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:50:57.909886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:57.911866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:57.970559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 239/475] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:50:57.985625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5c250-1ce4-4327-ae45-c1aae79d2d80 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:57.996955 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d98f6e-64b0-4f62-9237-2a8d7da82068 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:58.030580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e56e8-29ff-4e5e-ae27-71e94f4e2526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:58.031369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e56e8-29ff-4e5e-ae27-71e94f4e2526 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:58.031720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e56e8-29ff-4e5e-ae27-71e94f4e2526 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:58.032064 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e56e8-29ff-4e5e-ae27-71e94f4e2526 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:58.032862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e56e8-29ff-4e5e-ae27-71e94f4e2526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:58.033260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e56e8-29ff-4e5e-ae27-71e94f4e2526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:58.034843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e56e8-29ff-4e5e-ae27-71e94f4e2526 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:58.051154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 237/476] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:50:58.057561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e56e8-29ff-4e5e-ae27-71e94f4e2526 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:58.132801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d98f6e-64b0-4f62-9237-2a8d7da82068 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:58.173751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976fab0-2bdf-4cd8-8ea6-6a40f43255fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:58.174416 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976fab0-2bdf-4cd8-8ea6-6a40f43255fd None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:58.174743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976fab0-2bdf-4cd8-8ea6-6a40f43255fd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:58.175018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976fab0-2bdf-4cd8-8ea6-6a40f43255fd None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:58.175771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976fab0-2bdf-4cd8-8ea6-6a40f43255fd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:58.177322 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976fab0-2bdf-4cd8-8ea6-6a40f43255fd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:58.188493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e56e8-29ff-4e5e-ae27-71e94f4e2526 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:58.190854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.sql.core [None req-8976fab0-2bdf-4cd8-8ea6-6a40f43255fd None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 20:50:58.190854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 20:50:58.190854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [parameters: {'id': '181beda073c0446c8a882b3687579fed', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 20:50:58.190854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80269) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 20:50:58.204665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8976fab0-2bdf-4cd8-8ea6-6a40f43255fd None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() May 26 20:50:58.208734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return ref.to_dict() May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:50:58.211165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application yield resource May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application next(self.gen) May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.rollback() May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.force_reraise() May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise self.value May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application session.commit() May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 20:50:58.212735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._prepare_impl() May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.session.flush() May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._flush(objects) May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application compat.raise_( May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application flush_context.execute() May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rec.execute(self) May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 20:50:58.213904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:50:58.215003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application conn.query(q) May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application result.read() May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:50:58.217198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [parameters: {'id': '181beda073c0446c8a882b3687579fed', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:58.225605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 20:50:58.228327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:58.230385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 240/477] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 26 20:50:57 2022] POST /identity/v3/roles => generated 144 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 26 20:50:58.230385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976fab0-2bdf-4cd8-8ea6-6a40f43255fd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:58.230718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293bcf5-0694-4fe0-acc4-81982ada7056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:58.230718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293bcf5-0694-4fe0-acc4-81982ada7056 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:58.230718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293bcf5-0694-4fe0-acc4-81982ada7056 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:58.230718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b293bcf5-0694-4fe0-acc4-81982ada7056 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:58.231287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293bcf5-0694-4fe0-acc4-81982ada7056 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:58.234530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b293bcf5-0694-4fe0-acc4-81982ada7056 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:58.239302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:50:58.263618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 238/478] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:50:58 2022] POST /identity/v3/users => generated 249 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:50:58.341680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976fab0-2bdf-4cd8-8ea6-6a40f43255fd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:58.379265 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1d51b-ba54-4097-9831-f9274e3e86ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:58.380143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1d51b-ba54-4097-9831-f9274e3e86ab None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:58.380432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1d51b-ba54-4097-9831-f9274e3e86ab None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:58.380710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1d51b-ba54-4097-9831-f9274e3e86ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:58.393394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae1d51b-ba54-4097-9831-f9274e3e86ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1ae1d51b-ba54-4097-9831-f9274e3e86ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:50:58.401927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:50:58.404338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:50:58.406245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 241/479] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:50:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:50:58.411760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293bcf5-0694-4fe0-acc4-81982ada7056 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:50:58.534118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b293bcf5-0694-4fe0-acc4-81982ada7056 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:50:58.575899 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d41503-0a35-4dee-8a0a-5ad71deb2dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:50:58.576612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d41503-0a35-4dee-8a0a-5ad71deb2dba None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:50:58.576894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d41503-0a35-4dee-8a0a-5ad71deb2dba None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:50:58.577167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d41503-0a35-4dee-8a0a-5ad71deb2dba None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:50:58.577913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d41503-0a35-4dee-8a0a-5ad71deb2dba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:50:58.578358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d41503-0a35-4dee-8a0a-5ad71deb2dba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:50:58.579758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d41503-0a35-4dee-8a0a-5ad71deb2dba None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:50:58.589314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 239/480] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:50:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:00.133001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440d688-fe44-4221-9075-5e62f9895940 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:00.133001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440d688-fe44-4221-9075-5e62f9895940 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:00.133001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440d688-fe44-4221-9075-5e62f9895940 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:00.133001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 242/481] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:00.142325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b8909-e586-41e1-be26-0fb0eca1404b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:00.142325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b8909-e586-41e1-be26-0fb0eca1404b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:00.142325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b8909-e586-41e1-be26-0fb0eca1404b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:00.159950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-200b8909-e586-41e1-be26-0fb0eca1404b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:00.161458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-200b8909-e586-41e1-be26-0fb0eca1404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:00.190259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:00.247559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 240/482] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:00.295897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440d688-fe44-4221-9075-5e62f9895940 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:00.391010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:00.426136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440d688-fe44-4221-9075-5e62f9895940 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:00.459898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0766fd-b1ba-4ccf-8f87-6453aed3fc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:00.460698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0766fd-b1ba-4ccf-8f87-6453aed3fc09 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:00.460990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0766fd-b1ba-4ccf-8f87-6453aed3fc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:00.461273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0766fd-b1ba-4ccf-8f87-6453aed3fc09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:00.473261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0766fd-b1ba-4ccf-8f87-6453aed3fc09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6d0766fd-b1ba-4ccf-8f87-6453aed3fc09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:00.481375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:00.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:00.483080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:00.486577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 243/483] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:00.488999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b8909-e586-41e1-be26-0fb0eca1404b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:00.580224 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab958457-96f0-4d29-8ca6-95b4713251bd None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:00.580625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab958457-96f0-4d29-8ca6-95b4713251bd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:00.580918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab958457-96f0-4d29-8ca6-95b4713251bd None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:00.582418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 244/484] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:00.587832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40c84-fb9f-48a2-9f05-506abccb6cb0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:00.588347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40c84-fb9f-48a2-9f05-506abccb6cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:00.588835 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40c84-fb9f-48a2-9f05-506abccb6cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:00.612429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b8909-e586-41e1-be26-0fb0eca1404b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:00.624432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5db40c84-fb9f-48a2-9f05-506abccb6cb0 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:00.627753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5db40c84-fb9f-48a2-9f05-506abccb6cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:00.673569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc8d38-e3d1-4300-954a-f9821dc79e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:00.674599 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc8d38-e3d1-4300-954a-f9821dc79e46 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:00.675573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc8d38-e3d1-4300-954a-f9821dc79e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:00.676238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc8d38-e3d1-4300-954a-f9821dc79e46 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:00.678378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc8d38-e3d1-4300-954a-f9821dc79e46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:00.679618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc8d38-e3d1-4300-954a-f9821dc79e46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:00.681242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc8d38-e3d1-4300-954a-f9821dc79e46 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:00.707003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 241/485] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:00.710092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc8d38-e3d1-4300-954a-f9821dc79e46 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:00.748456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:00.758407 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 245/486] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:00.760746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fef2a2-e05a-4e56-9279-452322ed1449 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:00.761513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fef2a2-e05a-4e56-9279-452322ed1449 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:00.761997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fef2a2-e05a-4e56-9279-452322ed1449 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:00.763808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 246/487] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:00.776309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea0c92-3aac-4970-beff-7fe2ca2a57d4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:00.776899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea0c92-3aac-4970-beff-7fe2ca2a57d4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:00.777441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea0c92-3aac-4970-beff-7fe2ca2a57d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:00.796752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-55ea0c92-3aac-4970-beff-7fe2ca2a57d4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:00.798356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-55ea0c92-3aac-4970-beff-7fe2ca2a57d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:00.860644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc8d38-e3d1-4300-954a-f9821dc79e46 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:00.873423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 247/488] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:51:00.874742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea0c92-3aac-4970-beff-7fe2ca2a57d4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:00.878775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:00.920652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8ab3f-5ea0-4807-a13f-49e4bdf2833c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:00.921562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8ab3f-5ea0-4807-a13f-49e4bdf2833c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:00.921852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8ab3f-5ea0-4807-a13f-49e4bdf2833c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:00.922133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8ab3f-5ea0-4807-a13f-49e4bdf2833c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:00.937805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8ab3f-5ea0-4807-a13f-49e4bdf2833c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:00.939467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8ab3f-5ea0-4807-a13f-49e4bdf2833c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2ce8ab3f-5ea0-4807-a13f-49e4bdf2833c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:00.964264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:00.965278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:00.970984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:00.970984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:00.970984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:00.970984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:00.970984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 242/489] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:00.970984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8ab3f-5ea0-4807-a13f-49e4bdf2833c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:01.013972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea0c92-3aac-4970-beff-7fe2ca2a57d4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:01.055545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:01.065260 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954bc5e-3546-4924-a66d-d2774189e228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:01.066297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954bc5e-3546-4924-a66d-d2774189e228 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.066989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954bc5e-3546-4924-a66d-d2774189e228 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.067285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954bc5e-3546-4924-a66d-d2774189e228 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.081756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954bc5e-3546-4924-a66d-d2774189e228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:01.083606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8ab3f-5ea0-4807-a13f-49e4bdf2833c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-2954bc5e-3546-4924-a66d-d2774189e228 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:01.090968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:01.091914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:01.092737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:01.095593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 248/490] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:01.096643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954bc5e-3546-4924-a66d-d2774189e228 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:01.140124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4424df3-c06a-4065-aa50-1eed8eb29ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:01.141177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4424df3-c06a-4065-aa50-1eed8eb29ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.141470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4424df3-c06a-4065-aa50-1eed8eb29ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.141757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4424df3-c06a-4065-aa50-1eed8eb29ea8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.157066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4424df3-c06a-4065-aa50-1eed8eb29ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:01.168640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f4424df3-c06a-4065-aa50-1eed8eb29ea8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:01.169168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:01.170039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:01.170923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:01.174724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 243/491] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:01.176391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4424df3-c06a-4065-aa50-1eed8eb29ea8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:01.237223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954bc5e-3546-4924-a66d-d2774189e228 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:01.277933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8ab4a-c908-499e-a9d6-e0c1ab254e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:01.278977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8ab4a-c908-499e-a9d6-e0c1ab254e2a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.279493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8ab4a-c908-499e-a9d6-e0c1ab254e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.279976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8ab4a-c908-499e-a9d6-e0c1ab254e2a None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.280902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8ab4a-c908-499e-a9d6-e0c1ab254e2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:01.281424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8ab4a-c908-499e-a9d6-e0c1ab254e2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:01.283066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8ab4a-c908-499e-a9d6-e0c1ab254e2a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:01.296551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 249/492] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:01.298512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8ab4a-c908-499e-a9d6-e0c1ab254e2a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:01.303932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4424df3-c06a-4065-aa50-1eed8eb29ea8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:01.340113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084faa6-4a3d-4ef1-a521-38238b528ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:01.340787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084faa6-4a3d-4ef1-a521-38238b528ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.341076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084faa6-4a3d-4ef1-a521-38238b528ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.341358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9084faa6-4a3d-4ef1-a521-38238b528ca1 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.342102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084faa6-4a3d-4ef1-a521-38238b528ca1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:01.342413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084faa6-4a3d-4ef1-a521-38238b528ca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:01.343819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9084faa6-4a3d-4ef1-a521-38238b528ca1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:01.346799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:01.355219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 244/493] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:01.355381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084faa6-4a3d-4ef1-a521-38238b528ca1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:01.407435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8ab4a-c908-499e-a9d6-e0c1ab254e2a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:01.424316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:01.446852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfc4b0-c778-4464-bd7d-12e64a0e95a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:01.447623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfc4b0-c778-4464-bd7d-12e64a0e95a5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.447904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfc4b0-c778-4464-bd7d-12e64a0e95a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.448178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfc4b0-c778-4464-bd7d-12e64a0e95a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.448916 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfc4b0-c778-4464-bd7d-12e64a0e95a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:01.449222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfc4b0-c778-4464-bd7d-12e64a0e95a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:01.450644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfc4b0-c778-4464-bd7d-12e64a0e95a5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:01.458744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 250/494] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:01.459937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ebf30-9e36-41c0-a0e2-98ddbfb14c13 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.460225 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ebf30-9e36-41c0-a0e2-98ddbfb14c13 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.460504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ebf30-9e36-41c0-a0e2-98ddbfb14c13 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.461431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9084faa6-4a3d-4ef1-a521-38238b528ca1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:01.462407 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 251/495] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:01.463357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b686c-9287-48b9-9fea-32b172fcc844 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.463646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b686c-9287-48b9-9fea-32b172fcc844 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.463997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b686c-9287-48b9-9fea-32b172fcc844 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.465210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 252/496] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:01.465896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b686c-9287-48b9-9fea-32b172fcc844 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:01.500107 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79f621-07b9-4807-b974-232de886b351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:01.500851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f621-07b9-4807-b974-232de886b351 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.501122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f621-07b9-4807-b974-232de886b351 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.501403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f621-07b9-4807-b974-232de886b351 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.509015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79f621-07b9-4807-b974-232de886b351 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:01.510472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79f621-07b9-4807-b974-232de886b351 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ca79f621-07b9-4807-b974-232de886b351 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:01.526889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:01.527842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 245/497] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 26 20:51:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:01.533357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79f621-07b9-4807-b974-232de886b351 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:01.568442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b686c-9287-48b9-9fea-32b172fcc844 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:01.586188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:01.601975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac94bd5-441c-4fac-948b-d97936773500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:01.602841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac94bd5-441c-4fac-948b-d97936773500 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.603132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac94bd5-441c-4fac-948b-d97936773500 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.603421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac94bd5-441c-4fac-948b-d97936773500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.613703 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac94bd5-441c-4fac-948b-d97936773500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:01.615200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac94bd5-441c-4fac-948b-d97936773500 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:01.625875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79f621-07b9-4807-b974-232de886b351 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-eac94bd5-441c-4fac-948b-d97936773500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:01.632594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:01.634171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:01.635219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:01.635219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:01.635219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:01.635219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:01.635219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 253/498] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:01.635918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9d43d-2219-4504-9dd9-57ac8304a992 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.636024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9d43d-2219-4504-9dd9-57ac8304a992 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.636553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9d43d-2219-4504-9dd9-57ac8304a992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.652277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6ad9d43d-2219-4504-9dd9-57ac8304a992 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:01.653457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6ad9d43d-2219-4504-9dd9-57ac8304a992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:01.663353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036175d4-60d0-4abc-afd6-c237ffed5727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:01.664151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036175d4-60d0-4abc-afd6-c237ffed5727 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.664437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036175d4-60d0-4abc-afd6-c237ffed5727 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.664737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036175d4-60d0-4abc-afd6-c237ffed5727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.674742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036175d4-60d0-4abc-afd6-c237ffed5727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:01.676194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:01.676307 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036175d4-60d0-4abc-afd6-c237ffed5727 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-036175d4-60d0-4abc-afd6-c237ffed5727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:01.694466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:01.695433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:01.701518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:01.701518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:01.701518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:01.701518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:01.701518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 246/499] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:01.701518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185b9c9-0a62-4955-b88e-a36ee1ed12c1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.701518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185b9c9-0a62-4955-b88e-a36ee1ed12c1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.701518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185b9c9-0a62-4955-b88e-a36ee1ed12c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.714623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1185b9c9-0a62-4955-b88e-a36ee1ed12c1 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:01.715859 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1185b9c9-0a62-4955-b88e-a36ee1ed12c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:01.724968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 254/500] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:01.725838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9d43d-2219-4504-9dd9-57ac8304a992 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:01.784176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 247/501] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:01.784932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185b9c9-0a62-4955-b88e-a36ee1ed12c1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:01.831099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:01.844696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9d43d-2219-4504-9dd9-57ac8304a992 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:01.878003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bdc2f-2da1-42a5-b0cf-ade34832fdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:01.878728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bdc2f-2da1-42a5-b0cf-ade34832fdbb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.879008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bdc2f-2da1-42a5-b0cf-ade34832fdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.879283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bdc2f-2da1-42a5-b0cf-ade34832fdbb None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.880110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bdc2f-2da1-42a5-b0cf-ade34832fdbb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:01.880420 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bdc2f-2da1-42a5-b0cf-ade34832fdbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:01.881865 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bdc2f-2da1-42a5-b0cf-ade34832fdbb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:01.889433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185b9c9-0a62-4955-b88e-a36ee1ed12c1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:01.917621 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 255/502] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 26 20:51:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:01.918430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bdc2f-2da1-42a5-b0cf-ade34832fdbb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:01.921799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9adc1-c033-4998-be2e-5e3c49da8d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:01.922487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9adc1-c033-4998-be2e-5e3c49da8d2c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:01.922776 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9adc1-c033-4998-be2e-5e3c49da8d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:01.923057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9adc1-c033-4998-be2e-5e3c49da8d2c None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:01.923779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9adc1-c033-4998-be2e-5e3c49da8d2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:01.924087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9adc1-c033-4998-be2e-5e3c49da8d2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:01.925458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9adc1-c033-4998-be2e-5e3c49da8d2c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:01.937233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 248/503] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:01.937885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:01.938007 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9adc1-c033-4998-be2e-5e3c49da8d2c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.021442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bdc2f-2da1-42a5-b0cf-ade34832fdbb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.033439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:02.036818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9adc1-c033-4998-be2e-5e3c49da8d2c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.060554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ce3ac-58bf-49ef-96ae-cb9c74cedb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.061525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ce3ac-58bf-49ef-96ae-cb9c74cedb30 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.061812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ce3ac-58bf-49ef-96ae-cb9c74cedb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.062095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ce3ac-58bf-49ef-96ae-cb9c74cedb30 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.062858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ce3ac-58bf-49ef-96ae-cb9c74cedb30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:02.063167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ce3ac-58bf-49ef-96ae-cb9c74cedb30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:02.064575 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ce3ac-58bf-49ef-96ae-cb9c74cedb30 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:02.072877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767319c4-4b0e-45e3-99f8-c07c51846b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.073669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767319c4-4b0e-45e3-99f8-c07c51846b1f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.073952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767319c4-4b0e-45e3-99f8-c07c51846b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.074228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767319c4-4b0e-45e3-99f8-c07c51846b1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.078833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 256/504] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:02.079664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ce3ac-58bf-49ef-96ae-cb9c74cedb30 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.086038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767319c4-4b0e-45e3-99f8-c07c51846b1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-767319c4-4b0e-45e3-99f8-c07c51846b1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:02.093939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:02.094869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 249/505] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:02.095942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767319c4-4b0e-45e3-99f8-c07c51846b1f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.166016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:02.186770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ce3ac-58bf-49ef-96ae-cb9c74cedb30 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.196432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767319c4-4b0e-45e3-99f8-c07c51846b1f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.218479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d46a74-7405-4262-8048-3a20df28ccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.219274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d46a74-7405-4262-8048-3a20df28ccd9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.219558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d46a74-7405-4262-8048-3a20df28ccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.219828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d46a74-7405-4262-8048-3a20df28ccd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.229767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d840f4e-21c4-4fe7-8170-dbccc3a0ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.230407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d840f4e-21c4-4fe7-8170-dbccc3a0ec44 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.230532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d46a74-7405-4262-8048-3a20df28ccd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:02.230724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d840f4e-21c4-4fe7-8170-dbccc3a0ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.231002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d840f4e-21c4-4fe7-8170-dbccc3a0ec44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.231908 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d46a74-7405-4262-8048-3a20df28ccd9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d5d46a74-7405-4262-8048-3a20df28ccd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:02.249222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:02.250219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:02.251305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:02.251305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:02.251305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:02.251305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:02.251305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 257/506] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:02.251305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d46a74-7405-4262-8048-3a20df28ccd9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.261079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:02.301455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d840f4e-21c4-4fe7-8170-dbccc3a0ec44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:02.302968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d840f4e-21c4-4fe7-8170-dbccc3a0ec44 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:02.348617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d46a74-7405-4262-8048-3a20df28ccd9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.375943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:02.382082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e0840-7669-4fef-97cd-00c4259fc9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.382903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e0840-7669-4fef-97cd-00c4259fc9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.383190 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e0840-7669-4fef-97cd-00c4259fc9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.383494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e0840-7669-4fef-97cd-00c4259fc9e1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.391174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e0840-7669-4fef-97cd-00c4259fc9e1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:02.392719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e0840-7669-4fef-97cd-00c4259fc9e1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:02.399820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 250/507] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:02.400725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d840f4e-21c4-4fe7-8170-dbccc3a0ec44 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-130e0840-7669-4fef-97cd-00c4259fc9e1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:02.408907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:02.409962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:02.411070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 258/508] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:02.411762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e0840-7669-4fef-97cd-00c4259fc9e1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.475779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:02.524925 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e0840-7669-4fef-97cd-00c4259fc9e1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.526009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d840f4e-21c4-4fe7-8170-dbccc3a0ec44 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.563437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e5840-c4cf-4fda-ae74-7c4207627cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.564268 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e5840-c4cf-4fda-ae74-7c4207627cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.564382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a194-e469-4174-9450-179f35e52f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.564682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e5840-c4cf-4fda-ae74-7c4207627cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.564973 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a194-e469-4174-9450-179f35e52f22 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.565077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e5840-c4cf-4fda-ae74-7c4207627cc4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.565255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a194-e469-4174-9450-179f35e52f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.565535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5a194-e469-4174-9450-179f35e52f22 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.566336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5a194-e469-4174-9450-179f35e52f22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:02.566692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5a194-e469-4174-9450-179f35e52f22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:02.569095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5a194-e469-4174-9450-179f35e52f22 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:02.573001 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e5840-c4cf-4fda-ae74-7c4207627cc4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:02.574454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e5840-c4cf-4fda-ae74-7c4207627cc4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:02.577425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 259/509] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:02.578217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a194-e469-4174-9450-179f35e52f22 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b52e5840-c4cf-4fda-ae74-7c4207627cc4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:02.590837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:02.591989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 251/510] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:02.592927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e5840-c4cf-4fda-ae74-7c4207627cc4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.634957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:02.695594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5a194-e469-4174-9450-179f35e52f22 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.707195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e5840-c4cf-4fda-ae74-7c4207627cc4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.728778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:02.733136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30324d20-0ddc-44ed-8820-c5c9c594c16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.733797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30324d20-0ddc-44ed-8820-c5c9c594c16f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.734084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30324d20-0ddc-44ed-8820-c5c9c594c16f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.734364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30324d20-0ddc-44ed-8820-c5c9c594c16f None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.735122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30324d20-0ddc-44ed-8820-c5c9c594c16f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:02.735430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30324d20-0ddc-44ed-8820-c5c9c594c16f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:02.736839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30324d20-0ddc-44ed-8820-c5c9c594c16f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:02.742956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616d594-d9b6-4596-9e2c-6b431296c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.743740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616d594-d9b6-4596-9e2c-6b431296c30a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.744025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616d594-d9b6-4596-9e2c-6b431296c30a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.744309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616d594-d9b6-4596-9e2c-6b431296c30a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.749043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 260/511] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:02.749885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30324d20-0ddc-44ed-8820-c5c9c594c16f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.757498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616d594-d9b6-4596-9e2c-6b431296c30a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:02.760064 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616d594-d9b6-4596-9e2c-6b431296c30a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8616d594-d9b6-4596-9e2c-6b431296c30a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:02.779062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:02.780073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:02.781144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:02.781144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:02.781144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:02.781144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:02.781144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 252/512] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:02.781144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616d594-d9b6-4596-9e2c-6b431296c30a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.852658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30324d20-0ddc-44ed-8820-c5c9c594c16f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.865187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:02.877539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616d594-d9b6-4596-9e2c-6b431296c30a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:02.887673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e35c7-7147-47eb-b3b2-368414ef69a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.888340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e35c7-7147-47eb-b3b2-368414ef69a3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.888626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e35c7-7147-47eb-b3b2-368414ef69a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.888907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e35c7-7147-47eb-b3b2-368414ef69a3 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.889736 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e35c7-7147-47eb-b3b2-368414ef69a3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:02.890054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e35c7-7147-47eb-b3b2-368414ef69a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:02.891557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e35c7-7147-47eb-b3b2-368414ef69a3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:02.908610 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 261/513] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 26 20:51:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:02.909394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e35c7-7147-47eb-b3b2-368414ef69a3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.916334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3707734-eb33-4b8e-90ec-122beda9f476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:02.917129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3707734-eb33-4b8e-90ec-122beda9f476 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:02.917416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3707734-eb33-4b8e-90ec-122beda9f476 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:02.917697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3707734-eb33-4b8e-90ec-122beda9f476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:02.927639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3707734-eb33-4b8e-90ec-122beda9f476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:02.929085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3707734-eb33-4b8e-90ec-122beda9f476 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d3707734-eb33-4b8e-90ec-122beda9f476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:02.946614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:02.947443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:02.948506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:02.948506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:02.948506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:02.948506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:02.948506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 253/514] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:02.948506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3707734-eb33-4b8e-90ec-122beda9f476 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:02.973866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:03.008281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e35c7-7147-47eb-b3b2-368414ef69a3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:03.042256 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046f8e8-7e42-465d-96d8-dfb73bf8950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:03.042944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046f8e8-7e42-465d-96d8-dfb73bf8950f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:03.043235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046f8e8-7e42-465d-96d8-dfb73bf8950f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:03.043513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046f8e8-7e42-465d-96d8-dfb73bf8950f None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:03.044339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046f8e8-7e42-465d-96d8-dfb73bf8950f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:03.044652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046f8e8-7e42-465d-96d8-dfb73bf8950f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:03.046089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046f8e8-7e42-465d-96d8-dfb73bf8950f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:03.048132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3707734-eb33-4b8e-90ec-122beda9f476 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:03.065526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 262/515] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 26 20:51:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:03.066275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046f8e8-7e42-465d-96d8-dfb73bf8950f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:03.087998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78576569-1a7a-47a3-aa9a-8c35dfae77d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:03.088656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78576569-1a7a-47a3-aa9a-8c35dfae77d4 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:03.088938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78576569-1a7a-47a3-aa9a-8c35dfae77d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:03.089213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78576569-1a7a-47a3-aa9a-8c35dfae77d4 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:03.089901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78576569-1a7a-47a3-aa9a-8c35dfae77d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:03.091355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78576569-1a7a-47a3-aa9a-8c35dfae77d4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:03.093307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:03.111308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 254/516] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:51:02 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:51:03.112300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78576569-1a7a-47a3-aa9a-8c35dfae77d4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:03.186879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046f8e8-7e42-465d-96d8-dfb73bf8950f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:03.203402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:03.225692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78576569-1a7a-47a3-aa9a-8c35dfae77d4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:03.226416 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6ac4a-08d1-4c0f-af3d-8b9a8e0f8888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:03.227746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ac4a-08d1-4c0f-af3d-8b9a8e0f8888 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:03.228081 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ac4a-08d1-4c0f-af3d-8b9a8e0f8888 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:03.228310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6ac4a-08d1-4c0f-af3d-8b9a8e0f8888 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:03.229769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ac4a-08d1-4c0f-af3d-8b9a8e0f8888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:03.229769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ac4a-08d1-4c0f-af3d-8b9a8e0f8888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:03.230807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d6ac4a-08d1-4c0f-af3d-8b9a8e0f8888 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:03.243669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 263/517] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:03.248898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6ac4a-08d1-4c0f-af3d-8b9a8e0f8888 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:03.265918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f038a5-6599-4dda-bd5d-4ffb908c8870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:03.266749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f038a5-6599-4dda-bd5d-4ffb908c8870 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:03.267038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f038a5-6599-4dda-bd5d-4ffb908c8870 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:03.267327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f038a5-6599-4dda-bd5d-4ffb908c8870 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:03.346720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:03.347866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f038a5-6599-4dda-bd5d-4ffb908c8870 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:03.349534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f038a5-6599-4dda-bd5d-4ffb908c8870 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:03.362654 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6ac4a-08d1-4c0f-af3d-8b9a8e0f8888 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:03.400649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d08460-1c81-4390-93fc-6870ecbc1766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:03.401288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d08460-1c81-4390-93fc-6870ecbc1766 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:03.401569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d08460-1c81-4390-93fc-6870ecbc1766 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:03.401846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d08460-1c81-4390-93fc-6870ecbc1766 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:03.402597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d08460-1c81-4390-93fc-6870ecbc1766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:03.402905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d08460-1c81-4390-93fc-6870ecbc1766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:03.404294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d08460-1c81-4390-93fc-6870ecbc1766 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:03.416497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 264/518] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:03.417307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d08460-1c81-4390-93fc-6870ecbc1766 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:03.443155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:03.475789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 255/519] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:03.475789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f038a5-6599-4dda-bd5d-4ffb908c8870 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:03.525765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d08460-1c81-4390-93fc-6870ecbc1766 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:03.561675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7d4f0-d474-4f6a-a5ff-88d11356b578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:03.562346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7d4f0-d474-4f6a-a5ff-88d11356b578 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:03.562649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7d4f0-d474-4f6a-a5ff-88d11356b578 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:03.562928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7d4f0-d474-4f6a-a5ff-88d11356b578 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:03.578211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f038a5-6599-4dda-bd5d-4ffb908c8870 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:03.582961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:03.615994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5613f33-3790-4436-a6ed-5bbfa4e51364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:03.617577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5613f33-3790-4436-a6ed-5bbfa4e51364 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:03.618057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5613f33-3790-4436-a6ed-5bbfa4e51364 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:03.618577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5613f33-3790-4436-a6ed-5bbfa4e51364 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:03.628015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5613f33-3790-4436-a6ed-5bbfa4e51364 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:03.629589 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5613f33-3790-4436-a6ed-5bbfa4e51364 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:03.637093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7d4f0-d474-4f6a-a5ff-88d11356b578 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:03.640317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7d4f0-d474-4f6a-a5ff-88d11356b578 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b5613f33-3790-4436-a6ed-5bbfa4e51364 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:03.644663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:03.645693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:03.651788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:03.651788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:03.651788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:03.651788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:03.651788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:03.651788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:03.651788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:03.651788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:03.651788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 256/520] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:03.651788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5613f33-3790-4436-a6ed-5bbfa4e51364 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:03.689628 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:03.757654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5613f33-3790-4436-a6ed-5bbfa4e51364 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:03.759490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 265/521] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:03.760508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7d4f0-d474-4f6a-a5ff-88d11356b578 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:03.797083 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb7672-6d65-4edc-a294-adfab63da5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:03.797880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb7672-6d65-4edc-a294-adfab63da5cb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:03.798165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb7672-6d65-4edc-a294-adfab63da5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:03.798462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb7672-6d65-4edc-a294-adfab63da5cb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:03.812250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb7672-6d65-4edc-a294-adfab63da5cb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:03.813314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb7672-6d65-4edc-a294-adfab63da5cb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:03.815686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-52fb7672-6d65-4edc-a294-adfab63da5cb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:03.829334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:03.830503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:03.831664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 257/522] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:03.832236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb7672-6d65-4edc-a294-adfab63da5cb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:03.871273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7d4f0-d474-4f6a-a5ff-88d11356b578 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:03.905828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018109c-3e7b-4311-a850-6e4f478d9cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:03.906638 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018109c-3e7b-4311-a850-6e4f478d9cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:03.906923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018109c-3e7b-4311-a850-6e4f478d9cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:03.907194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018109c-3e7b-4311-a850-6e4f478d9cb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:03.917614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018109c-3e7b-4311-a850-6e4f478d9cb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:03.919107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018109c-3e7b-4311-a850-6e4f478d9cb8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3018109c-3e7b-4311-a850-6e4f478d9cb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:03.937142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:03.938053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:03.938959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:03.938959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:03.938959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:03.938959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:03.939182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:03.939254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 266/523] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:03.939606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018109c-3e7b-4311-a850-6e4f478d9cb8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:03.945886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb7672-6d65-4edc-a294-adfab63da5cb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:03.982927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55641a96-921f-4e50-a689-0ff088c4c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:03.983591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55641a96-921f-4e50-a689-0ff088c4c33c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:03.983873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55641a96-921f-4e50-a689-0ff088c4c33c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:03.984152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55641a96-921f-4e50-a689-0ff088c4c33c None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:03.984928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55641a96-921f-4e50-a689-0ff088c4c33c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:03.985237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55641a96-921f-4e50-a689-0ff088c4c33c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:03.986798 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55641a96-921f-4e50-a689-0ff088c4c33c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:04.005835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 258/524] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:04.006726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55641a96-921f-4e50-a689-0ff088c4c33c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:04.052844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:04.056043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018109c-3e7b-4311-a850-6e4f478d9cb8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:04.090628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0e1a3-5d6f-428f-a232-3ef0b1326c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:04.091426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0e1a3-5d6f-428f-a232-3ef0b1326c98 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:04.091713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0e1a3-5d6f-428f-a232-3ef0b1326c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:04.091997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0e1a3-5d6f-428f-a232-3ef0b1326c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:04.101824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0e1a3-5d6f-428f-a232-3ef0b1326c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:04.103898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0e1a3-5d6f-428f-a232-3ef0b1326c98 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:04.115334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55641a96-921f-4e50-a689-0ff088c4c33c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-aeb0e1a3-5d6f-428f-a232-3ef0b1326c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:04.121344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:04.122361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:04.123237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:04.123237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:04.123237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:04.123237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:04.123237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 267/525] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:04.123237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0e1a3-5d6f-428f-a232-3ef0b1326c98 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:04.156251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36953823-a1de-44d0-aa69-4d293828f99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:04.157226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36953823-a1de-44d0-aa69-4d293828f99d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:04.157745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36953823-a1de-44d0-aa69-4d293828f99d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:04.158171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36953823-a1de-44d0-aa69-4d293828f99d None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:04.159170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36953823-a1de-44d0-aa69-4d293828f99d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:04.159726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36953823-a1de-44d0-aa69-4d293828f99d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:04.161410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36953823-a1de-44d0-aa69-4d293828f99d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:04.200429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 259/526] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 26 20:51:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:04.201645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36953823-a1de-44d0-aa69-4d293828f99d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:04.221665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:04.230349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0e1a3-5d6f-428f-a232-3ef0b1326c98 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:04.267834 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93495fe3-cff2-44c4-91ba-73bd3bac80ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:04.268637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93495fe3-cff2-44c4-91ba-73bd3bac80ab None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:04.268918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93495fe3-cff2-44c4-91ba-73bd3bac80ab None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:04.269304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93495fe3-cff2-44c4-91ba-73bd3bac80ab None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:04.270027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93495fe3-cff2-44c4-91ba-73bd3bac80ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:04.270484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93495fe3-cff2-44c4-91ba-73bd3bac80ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:04.272125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93495fe3-cff2-44c4-91ba-73bd3bac80ab None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:04.287917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 268/527] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:04.287917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93495fe3-cff2-44c4-91ba-73bd3bac80ab None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:04.301438 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:04.316125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36953823-a1de-44d0-aa69-4d293828f99d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:04.351849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cc2ac-b319-4e5d-9960-b69e7db9ac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:04.353032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cc2ac-b319-4e5d-9960-b69e7db9ac63 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:04.353320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cc2ac-b319-4e5d-9960-b69e7db9ac63 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:04.353594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cc2ac-b319-4e5d-9960-b69e7db9ac63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:04.354426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cc2ac-b319-4e5d-9960-b69e7db9ac63 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=490d773bf21446789b77c5793a518a5a) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:04.354770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cc2ac-b319-4e5d-9960-b69e7db9ac63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:04.356218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cc2ac-b319-4e5d-9960-b69e7db9ac63 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:04.370475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 260/528] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:04 2022] GET /identity/v3/role_assignments?user.id=490d773bf21446789b77c5793a518a5a&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:04.370475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cc2ac-b319-4e5d-9960-b69e7db9ac63 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:04.409828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93495fe3-cff2-44c4-91ba-73bd3bac80ab None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:04.449751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e50c5a-2dab-479e-bb80-d5f166da2ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:04.450735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e50c5a-2dab-479e-bb80-d5f166da2ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:04.451465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e50c5a-2dab-479e-bb80-d5f166da2ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:04.451800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e50c5a-2dab-479e-bb80-d5f166da2ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:04.452864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e50c5a-2dab-479e-bb80-d5f166da2ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:04.453182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e50c5a-2dab-479e-bb80-d5f166da2ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:04.454938 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e50c5a-2dab-479e-bb80-d5f166da2ad6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:04.472903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 269/529] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:04.473811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e50c5a-2dab-479e-bb80-d5f166da2ad6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:04.481310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:04.506925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cc2ac-b319-4e5d-9960-b69e7db9ac63 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:04.554307 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00907ef-9a51-4da2-83ce-5df03915cf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:04.555070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00907ef-9a51-4da2-83ce-5df03915cf7e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:04.555964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00907ef-9a51-4da2-83ce-5df03915cf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:04.556241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00907ef-9a51-4da2-83ce-5df03915cf7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:04.581551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:04.613015 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e50c5a-2dab-479e-bb80-d5f166da2ad6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:04.646689 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00907ef-9a51-4da2-83ce-5df03915cf7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:04.648149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00907ef-9a51-4da2-83ce-5df03915cf7e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:04.662823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6314193-b586-43b1-8085-20c6795d666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:04.664388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6314193-b586-43b1-8085-20c6795d666f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:04.665670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6314193-b586-43b1-8085-20c6795d666f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:04.666170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6314193-b586-43b1-8085-20c6795d666f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:04.675911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6314193-b586-43b1-8085-20c6795d666f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:04.678512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6314193-b586-43b1-8085-20c6795d666f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e6314193-b586-43b1-8085-20c6795d666f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:04.697952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:04.698876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:04.699737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:04.699737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:04.699737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:04.699737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:04.699737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:04.699737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:04.699737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:04.699737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:04.700902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 270/530] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:04.700902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6314193-b586-43b1-8085-20c6795d666f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:04.737844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:04.763255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 261/531] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:04.764124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00907ef-9a51-4da2-83ce-5df03915cf7e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:04.833339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6314193-b586-43b1-8085-20c6795d666f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:04.865034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:04.871644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b489b7f-40e4-4500-8265-dec21bd916a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:04.872688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b489b7f-40e4-4500-8265-dec21bd916a4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:04.872800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b489b7f-40e4-4500-8265-dec21bd916a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:04.873056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b489b7f-40e4-4500-8265-dec21bd916a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:04.880283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b489b7f-40e4-4500-8265-dec21bd916a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:04.882188 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b489b7f-40e4-4500-8265-dec21bd916a4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1b489b7f-40e4-4500-8265-dec21bd916a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:04.899661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:04.900564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:04.901437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:04.901437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:04.901437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:04.901437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:04.901437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:04.901437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:04.901437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:04.901437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:04.901437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 271/532] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:04.901904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b489b7f-40e4-4500-8265-dec21bd916a4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:04.910400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00907ef-9a51-4da2-83ce-5df03915cf7e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:04.952817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf2c12-e25a-4065-89c5-f7ef39683f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:04.953396 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf2c12-e25a-4065-89c5-f7ef39683f08 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:04.953680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf2c12-e25a-4065-89c5-f7ef39683f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:04.953954 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf2c12-e25a-4065-89c5-f7ef39683f08 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:04.954777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf2c12-e25a-4065-89c5-f7ef39683f08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:04.955084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf2c12-e25a-4065-89c5-f7ef39683f08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:04.956512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf2c12-e25a-4065-89c5-f7ef39683f08 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:04.973912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 262/533] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:04.974825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf2c12-e25a-4065-89c5-f7ef39683f08 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:05.022829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:05.036819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b489b7f-40e4-4500-8265-dec21bd916a4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:05.077453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c04a4f-6703-4e59-83a3-0b05b8ee3a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:05.078368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c04a4f-6703-4e59-83a3-0b05b8ee3a59 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:05.078757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c04a4f-6703-4e59-83a3-0b05b8ee3a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:05.079108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c04a4f-6703-4e59-83a3-0b05b8ee3a59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:05.090285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c04a4f-6703-4e59-83a3-0b05b8ee3a59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:05.091848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c04a4f-6703-4e59-83a3-0b05b8ee3a59 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:05.097062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf2c12-e25a-4065-89c5-f7ef39683f08 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c7c04a4f-6703-4e59-83a3-0b05b8ee3a59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:05.110062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:05.111026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:05.111872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:05.111872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:05.111872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:05.111872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:05.111872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 272/534] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:05.112146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c04a4f-6703-4e59-83a3-0b05b8ee3a59 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:05.144536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd3268-cc84-459e-a8ef-f377a691fde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:05.146182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd3268-cc84-459e-a8ef-f377a691fde3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:05.146708 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd3268-cc84-459e-a8ef-f377a691fde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:05.147169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffd3268-cc84-459e-a8ef-f377a691fde3 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:05.148166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd3268-cc84-459e-a8ef-f377a691fde3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:05.148912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd3268-cc84-459e-a8ef-f377a691fde3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:05.151885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffd3268-cc84-459e-a8ef-f377a691fde3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:05.166088 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:05.168833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 263/535] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:05.169736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd3268-cc84-459e-a8ef-f377a691fde3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:05.237985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c04a4f-6703-4e59-83a3-0b05b8ee3a59 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:05.274973 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51f426-c9ba-4b07-9b77-99fe3c925635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:05.275653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51f426-c9ba-4b07-9b77-99fe3c925635 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:05.275935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51f426-c9ba-4b07-9b77-99fe3c925635 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:05.276211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51f426-c9ba-4b07-9b77-99fe3c925635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:05.277053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51f426-c9ba-4b07-9b77-99fe3c925635 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=afe351334485400a8c11f17492c9ff5a) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:05.277436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51f426-c9ba-4b07-9b77-99fe3c925635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:05.278816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51f426-c9ba-4b07-9b77-99fe3c925635 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:05.290747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 273/536] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:05 2022] GET /identity/v3/role_assignments?user.id=afe351334485400a8c11f17492c9ff5a&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:05.291740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51f426-c9ba-4b07-9b77-99fe3c925635 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:05.299854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffd3268-cc84-459e-a8ef-f377a691fde3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:05.310869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:05.344054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38935103-8a62-41b2-9516-18d7c4257607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:05.344962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38935103-8a62-41b2-9516-18d7c4257607 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:05.345525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38935103-8a62-41b2-9516-18d7c4257607 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:05.345923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38935103-8a62-41b2-9516-18d7c4257607 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:05.346832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38935103-8a62-41b2-9516-18d7c4257607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:05.347259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38935103-8a62-41b2-9516-18d7c4257607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:05.348804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38935103-8a62-41b2-9516-18d7c4257607 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:05.363133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 264/537] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:05.369120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38935103-8a62-41b2-9516-18d7c4257607 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:05.500105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51f426-c9ba-4b07-9b77-99fe3c925635 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:05.515412 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:05.523768 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38935103-8a62-41b2-9516-18d7c4257607 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:05.552482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca2eaad-9e6c-445d-a9db-7027317208e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:05.553578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2eaad-9e6c-445d-a9db-7027317208e6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:05.554540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2eaad-9e6c-445d-a9db-7027317208e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:05.554864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca2eaad-9e6c-445d-a9db-7027317208e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:05.556523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2eaad-9e6c-445d-a9db-7027317208e6 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=2c3bd8dd462846bf9423f8528aa5c489) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:05.556844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2eaad-9e6c-445d-a9db-7027317208e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:05.558968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca2eaad-9e6c-445d-a9db-7027317208e6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:05.572588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 274/538] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:05 2022] GET /identity/v3/role_assignments?user.id=2c3bd8dd462846bf9423f8528aa5c489&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:05.574519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fe556-3621-4839-ab1e-87f69881ea04 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:05.575648 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fe556-3621-4839-ab1e-87f69881ea04 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:05.576139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fe556-3621-4839-ab1e-87f69881ea04 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:05.583002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 275/539] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:05.586076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46c175-8487-44d0-aad4-27823770aeb5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:05.588071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a8fdc-91fc-40c0-9ea0-00398933757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:05.589193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a8fdc-91fc-40c0-9ea0-00398933757a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:05.589684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a8fdc-91fc-40c0-9ea0-00398933757a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:05.590158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a8fdc-91fc-40c0-9ea0-00398933757a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:05.591722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46c175-8487-44d0-aad4-27823770aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:05.593486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46c175-8487-44d0-aad4-27823770aeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:05.605399 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a8fdc-91fc-40c0-9ea0-00398933757a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:05.608397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a8fdc-91fc-40c0-9ea0-00398933757a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-103a8fdc-91fc-40c0-9ea0-00398933757a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:05.637697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:05.638681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:05.639520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:05.639520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:05.639520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:05.639520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:05.639520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:05.639520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:05.639520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:05.639520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:05.641598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9d46c175-8487-44d0-aad4-27823770aeb5 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:05.643086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9d46c175-8487-44d0-aad4-27823770aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:05.651151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 265/540] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:05.657757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a8fdc-91fc-40c0-9ea0-00398933757a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:05.713317 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:05.776427 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 276/541] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:05.817388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46c175-8487-44d0-aad4-27823770aeb5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:05.852946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a8fdc-91fc-40c0-9ea0-00398933757a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:05.895354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f2844-0aa1-4b66-9188-462ec8e41cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:05.897964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f2844-0aa1-4b66-9188-462ec8e41cea None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:05.898464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f2844-0aa1-4b66-9188-462ec8e41cea None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:05.898968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f2844-0aa1-4b66-9188-462ec8e41cea None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:05.901570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f2844-0aa1-4b66-9188-462ec8e41cea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:05.902088 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f2844-0aa1-4b66-9188-462ec8e41cea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:05.905033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f2844-0aa1-4b66-9188-462ec8e41cea None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:05.929571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 266/542] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:05.935861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f2844-0aa1-4b66-9188-462ec8e41cea None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:06.006798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46c175-8487-44d0-aad4-27823770aeb5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:06.008759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:06.058804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:06.077570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7731bbb-8001-4060-ba82-d86191cf9115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:06.078382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7731bbb-8001-4060-ba82-d86191cf9115 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:06.079385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7731bbb-8001-4060-ba82-d86191cf9115 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:06.079667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7731bbb-8001-4060-ba82-d86191cf9115 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:06.092519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7731bbb-8001-4060-ba82-d86191cf9115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:06.093666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f2844-0aa1-4b66-9188-462ec8e41cea None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e7731bbb-8001-4060-ba82-d86191cf9115 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:06.100512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:06.101405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:06.103183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 277/543] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:06.108317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7731bbb-8001-4060-ba82-d86191cf9115 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:06.144825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c931be-ca27-4bf7-876e-43ec328c1e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:06.146287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c931be-ca27-4bf7-876e-43ec328c1e6f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:06.146629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c931be-ca27-4bf7-876e-43ec328c1e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:06.146916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c931be-ca27-4bf7-876e-43ec328c1e6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:06.147760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c931be-ca27-4bf7-876e-43ec328c1e6f None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=94b8d8b089264b5192a651b41bf2a205) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:06.148074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c931be-ca27-4bf7-876e-43ec328c1e6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:06.150062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c931be-ca27-4bf7-876e-43ec328c1e6f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:06.156178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 267/544] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:05 2022] GET /identity/v3/role_assignments?user.id=94b8d8b089264b5192a651b41bf2a205&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:06.260870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7731bbb-8001-4060-ba82-d86191cf9115 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:06.299127 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15164c-84b7-46a1-aee3-bba9822b94f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:06.299812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15164c-84b7-46a1-aee3-bba9822b94f0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:06.300100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15164c-84b7-46a1-aee3-bba9822b94f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:06.300381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15164c-84b7-46a1-aee3-bba9822b94f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:06.301143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15164c-84b7-46a1-aee3-bba9822b94f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:06.301451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15164c-84b7-46a1-aee3-bba9822b94f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:06.302922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f15164c-84b7-46a1-aee3-bba9822b94f0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:06.312834 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 278/545] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:06.319866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c931be-ca27-4bf7-876e-43ec328c1e6f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:06.442354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c931be-ca27-4bf7-876e-43ec328c1e6f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:06.477288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4605417-c5e0-40d8-88e4-826dafd9f990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:06.478086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4605417-c5e0-40d8-88e4-826dafd9f990 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:06.478371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4605417-c5e0-40d8-88e4-826dafd9f990 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:06.478681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4605417-c5e0-40d8-88e4-826dafd9f990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:06.489580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4605417-c5e0-40d8-88e4-826dafd9f990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:06.491067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4605417-c5e0-40d8-88e4-826dafd9f990 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:06.499161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c4605417-c5e0-40d8-88e4-826dafd9f990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:06.509224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:06.510509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:06.512952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:06.512952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:06.512952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:06.512952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:06.512952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 268/546] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:06.519588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15164c-84b7-46a1-aee3-bba9822b94f0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:06.578469 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:06.698616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15164c-84b7-46a1-aee3-bba9822b94f0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:06.745587 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e55618-e823-44a9-9d50-9f8029f5c0c6 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:06.745888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e55618-e823-44a9-9d50-9f8029f5c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:06.746178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e55618-e823-44a9-9d50-9f8029f5c0c6 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:06.746863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc2b61-1187-46b5-9bee-850a78d8aa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:06.747681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc2b61-1187-46b5-9bee-850a78d8aa61 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:06.747782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 269/547] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:06.747867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc2b61-1187-46b5-9bee-850a78d8aa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:06.748964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc2b61-1187-46b5-9bee-850a78d8aa61 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:06.748964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc2b61-1187-46b5-9bee-850a78d8aa61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:06.749122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc2b61-1187-46b5-9bee-850a78d8aa61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:06.750596 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc2b61-1187-46b5-9bee-850a78d8aa61 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:06.753709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35adb2f-d2b6-4981-8fbf-ca8979dfb5fa None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:06.754210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35adb2f-d2b6-4981-8fbf-ca8979dfb5fa None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:06.754733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35adb2f-d2b6-4981-8fbf-ca8979dfb5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:06.772115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 279/548] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:06.772620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e35adb2f-d2b6-4981-8fbf-ca8979dfb5fa None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:06.773883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e35adb2f-d2b6-4981-8fbf-ca8979dfb5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:06.779820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc2b61-1187-46b5-9bee-850a78d8aa61 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:06.890180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:06.911558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc2b61-1187-46b5-9bee-850a78d8aa61 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:06.926308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 270/549] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:51:06.950547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ff204-768a-4d2a-be26-e3ecc21f0413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:06.951110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35adb2f-d2b6-4981-8fbf-ca8979dfb5fa None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:06.951921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ff204-768a-4d2a-be26-e3ecc21f0413 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:06.952401 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ff204-768a-4d2a-be26-e3ecc21f0413 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:06.952895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ff204-768a-4d2a-be26-e3ecc21f0413 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:06.968796 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ff204-768a-4d2a-be26-e3ecc21f0413 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:06.978468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ff204-768a-4d2a-be26-e3ecc21f0413 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b49ff204-768a-4d2a-be26-e3ecc21f0413 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:07.011573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:07.013050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:07.014122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 280/550] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:07.018413 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ff204-768a-4d2a-be26-e3ecc21f0413 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:07.068611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35adb2f-d2b6-4981-8fbf-ca8979dfb5fa None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:07.106469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:07.112521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3505ac-7255-4c4b-ba06-421ced981e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:07.113389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3505ac-7255-4c4b-ba06-421ced981e1a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:07.113720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3505ac-7255-4c4b-ba06-421ced981e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:07.114000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3505ac-7255-4c4b-ba06-421ced981e1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:07.131712 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3505ac-7255-4c4b-ba06-421ced981e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-dd3505ac-7255-4c4b-ba06-421ced981e1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:07.139892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:07.140857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:07.141851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 271/551] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:07.149332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3505ac-7255-4c4b-ba06-421ced981e1a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:07.241662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ff204-768a-4d2a-be26-e3ecc21f0413 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:07.310411 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70097c89-f7e3-498f-a85e-211d3df2ae4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:07.311888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70097c89-f7e3-498f-a85e-211d3df2ae4e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:07.312371 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70097c89-f7e3-498f-a85e-211d3df2ae4e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:07.312845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70097c89-f7e3-498f-a85e-211d3df2ae4e None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:07.313876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70097c89-f7e3-498f-a85e-211d3df2ae4e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:07.314409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70097c89-f7e3-498f-a85e-211d3df2ae4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:07.316071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70097c89-f7e3-498f-a85e-211d3df2ae4e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:07.332209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3505ac-7255-4c4b-ba06-421ced981e1a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:07.351177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 281/552] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 26 20:51:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:07.356263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70097c89-f7e3-498f-a85e-211d3df2ae4e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:07.366847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:07.411310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b219f-a6a0-48a1-b826-ac05ec289fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:07.412367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b219f-a6a0-48a1-b826-ac05ec289fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:07.412870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b219f-a6a0-48a1-b826-ac05ec289fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:07.413364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b219f-a6a0-48a1-b826-ac05ec289fc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:07.414308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b219f-a6a0-48a1-b826-ac05ec289fc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:07.416036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b219f-a6a0-48a1-b826-ac05ec289fc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:07.417741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b219f-a6a0-48a1-b826-ac05ec289fc8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:07.441031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 272/553] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:07.456487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b219f-a6a0-48a1-b826-ac05ec289fc8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:07.502691 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70097c89-f7e3-498f-a85e-211d3df2ae4e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:07.539544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e643f-c47f-4e1b-95a6-b7a81c0e3291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:07.540589 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e643f-c47f-4e1b-95a6-b7a81c0e3291 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:07.541072 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e643f-c47f-4e1b-95a6-b7a81c0e3291 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:07.541570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e643f-c47f-4e1b-95a6-b7a81c0e3291 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:07.552916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:07.634975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e643f-c47f-4e1b-95a6-b7a81c0e3291 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:07.635309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b219f-a6a0-48a1-b826-ac05ec289fc8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:07.637398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e643f-c47f-4e1b-95a6-b7a81c0e3291 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:07.650820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:07.677151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d2f11-a5a3-430d-abb5-56acf76ef778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:07.677883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d2f11-a5a3-430d-abb5-56acf76ef778 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:07.678168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d2f11-a5a3-430d-abb5-56acf76ef778 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:07.678472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d2f11-a5a3-430d-abb5-56acf76ef778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:07.705900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d2f11-a5a3-430d-abb5-56acf76ef778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:07.707768 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013d2f11-a5a3-430d-abb5-56acf76ef778 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-013d2f11-a5a3-430d-abb5-56acf76ef778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:07.752797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:07.753935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:07.755058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:07.755058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:07.755058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:07.755058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:07.757563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 273/554] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:07.761608 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d2f11-a5a3-430d-abb5-56acf76ef778 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:07.864888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 282/555] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 20:51:07.871722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e643f-c47f-4e1b-95a6-b7a81c0e3291 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:07.942359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013d2f11-a5a3-430d-abb5-56acf76ef778 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:07.952528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:07.985765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e643f-c47f-4e1b-95a6-b7a81c0e3291 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:08.014927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:08.027191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d1a84-f9b0-4c3a-a50d-16819556ed82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.029022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d1a84-f9b0-4c3a-a50d-16819556ed82 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.029319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d1a84-f9b0-4c3a-a50d-16819556ed82 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.029601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d1a84-f9b0-4c3a-a50d-16819556ed82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.044259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb9966-7c0d-4d00-af1f-ec97d4199f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.044934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb9966-7c0d-4d00-af1f-ec97d4199f9a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.045216 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb9966-7c0d-4d00-af1f-ec97d4199f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.045489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bb9966-7c0d-4d00-af1f-ec97d4199f9a None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.047756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb9966-7c0d-4d00-af1f-ec97d4199f9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:08.048077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb9966-7c0d-4d00-af1f-ec97d4199f9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:08.049462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bb9966-7c0d-4d00-af1f-ec97d4199f9a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:08.059979 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d1a84-f9b0-4c3a-a50d-16819556ed82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:08.061623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d1a84-f9b0-4c3a-a50d-16819556ed82 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:08.078517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 274/556] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:08.082929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb9966-7c0d-4d00-af1f-ec97d4199f9a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-288d1a84-f9b0-4c3a-a50d-16819556ed82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:08.103431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:08.104805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:08.105686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:08.105686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:08.105686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:08.105686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:08.106366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 283/557] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:08.112848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d1a84-f9b0-4c3a-a50d-16819556ed82 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:08.206928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bb9966-7c0d-4d00-af1f-ec97d4199f9a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:08.227960 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d1a84-f9b0-4c3a-a50d-16819556ed82 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:08.245225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e11f3-2c01-497d-833c-6a3cabceb1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.246136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e11f3-2c01-497d-833c-6a3cabceb1fb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.246424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e11f3-2c01-497d-833c-6a3cabceb1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.246733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e11f3-2c01-497d-833c-6a3cabceb1fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.254584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e11f3-2c01-497d-833c-6a3cabceb1fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:08.256187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e11f3-2c01-497d-833c-6a3cabceb1fb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:08.263128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55d20d-b4e5-4ae4-ad11-16b37bfc9052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.263788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55d20d-b4e5-4ae4-ad11-16b37bfc9052 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.264076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55d20d-b4e5-4ae4-ad11-16b37bfc9052 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.264354 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55d20d-b4e5-4ae4-ad11-16b37bfc9052 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.265074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55d20d-b4e5-4ae4-ad11-16b37bfc9052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:08.265384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55d20d-b4e5-4ae4-ad11-16b37bfc9052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:08.266791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55d20d-b4e5-4ae4-ad11-16b37bfc9052 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b09e11f3-2c01-497d-833c-6a3cabceb1fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:08.271522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:08.272516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:08.273437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 275/558] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:08.274171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ec42d-5cb0-49e9-a1aa-94fd9d27189c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.278676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ec42d-5cb0-49e9-a1aa-94fd9d27189c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.278676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ec42d-5cb0-49e9-a1aa-94fd9d27189c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.278676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 276/559] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:08.281263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f4614-7261-477e-8630-57ab57676145 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.281559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f4614-7261-477e-8630-57ab57676145 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.281845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f4614-7261-477e-8630-57ab57676145 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.287416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 277/560] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:08.287543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 284/561] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:08.287543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f0613-0566-413c-bb40-6999111998b6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.287543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f0613-0566-413c-bb40-6999111998b6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.287543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f0613-0566-413c-bb40-6999111998b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.288255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f4614-7261-477e-8630-57ab57676145 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:08.303325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-797f0613-0566-413c-bb40-6999111998b6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:08.304522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-797f0613-0566-413c-bb40-6999111998b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:08.314593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:08.315549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:08.376055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 285/562] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:08.377696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4236171-0495-4e16-bc0c-0a298e7a3338 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.378049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4236171-0495-4e16-bc0c-0a298e7a3338 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.378418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4236171-0495-4e16-bc0c-0a298e7a3338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.394819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c4236171-0495-4e16-bc0c-0a298e7a3338 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:08.396073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c4236171-0495-4e16-bc0c-0a298e7a3338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:08.418818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f4614-7261-477e-8630-57ab57676145 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:08.458195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1eae1-599c-4df4-8ff9-b0fdee0c0e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.458907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1eae1-599c-4df4-8ff9-b0fdee0c0e11 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.459209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1eae1-599c-4df4-8ff9-b0fdee0c0e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.459492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1eae1-599c-4df4-8ff9-b0fdee0c0e11 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.460321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1eae1-599c-4df4-8ff9-b0fdee0c0e11 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:08.460630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1eae1-599c-4df4-8ff9-b0fdee0c0e11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:08.462718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1eae1-599c-4df4-8ff9-b0fdee0c0e11 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:08.463802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 286/563] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:08.475339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4236171-0495-4e16-bc0c-0a298e7a3338 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:08.485221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 278/564] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:08.486112 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1eae1-599c-4df4-8ff9-b0fdee0c0e11 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:08.561809 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:08.574730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:08.589308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4236171-0495-4e16-bc0c-0a298e7a3338 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:08.596237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1eae1-599c-4df4-8ff9-b0fdee0c0e11 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:08.627119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137b76a-6eea-4f7a-bf23-f4a819787888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.628016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137b76a-6eea-4f7a-bf23-f4a819787888 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.628471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137b76a-6eea-4f7a-bf23-f4a819787888 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.628681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137b76a-6eea-4f7a-bf23-f4a819787888 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.632978 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d138e-900a-4c49-b24e-db7fe7361a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.633820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d138e-900a-4c49-b24e-db7fe7361a8a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.634109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d138e-900a-4c49-b24e-db7fe7361a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.634387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d138e-900a-4c49-b24e-db7fe7361a8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.635607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137b76a-6eea-4f7a-bf23-f4a819787888 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:08.637391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137b76a-6eea-4f7a-bf23-f4a819787888 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:08.645445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d138e-900a-4c49-b24e-db7fe7361a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f137b76a-6eea-4f7a-bf23-f4a819787888 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:08.652039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:08.652911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:08.653820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:08.653820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:08.653820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:08.653820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:08.653820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:08.653820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:08.653820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:08.653820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:08.653820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 287/565] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:08.654319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137b76a-6eea-4f7a-bf23-f4a819787888 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2f5d138e-900a-4c49-b24e-db7fe7361a8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:08.658305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:08.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:08.660153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 279/566] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:08.660848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d138e-900a-4c49-b24e-db7fe7361a8a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:08.759228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137b76a-6eea-4f7a-bf23-f4a819787888 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:08.766468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d138e-900a-4c49-b24e-db7fe7361a8a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:08.791703 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4178d6-dcf2-4884-91f5-57d745d662b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.792495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4178d6-dcf2-4884-91f5-57d745d662b7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.792779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4178d6-dcf2-4884-91f5-57d745d662b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.793055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4178d6-dcf2-4884-91f5-57d745d662b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.794970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:08.800390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b665c-f26f-4f80-ace2-4c575fb32180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.801185 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b665c-f26f-4f80-ace2-4c575fb32180 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.801467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b665c-f26f-4f80-ace2-4c575fb32180 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.801741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b665c-f26f-4f80-ace2-4c575fb32180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.807275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4178d6-dcf2-4884-91f5-57d745d662b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:08.810657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:08.814641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b665c-f26f-4f80-ace2-4c575fb32180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3e4178d6-dcf2-4884-91f5-57d745d662b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:08.815397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:08.816315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:08.817157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:08.817829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b665c-f26f-4f80-ace2-4c575fb32180 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:08.817900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 288/567] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:08.817900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4178d6-dcf2-4884-91f5-57d745d662b7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-d85b665c-f26f-4f80-ace2-4c575fb32180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:08.834467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:08.835372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:08.836258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:08.836258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:08.836258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:08.836258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:08.836258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 280/568] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:08.837585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b665c-f26f-4f80-ace2-4c575fb32180 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:08.919214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4178d6-dcf2-4884-91f5-57d745d662b7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:08.934362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b665c-f26f-4f80-ace2-4c575fb32180 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:08.951980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92770892-2d55-402e-903d-3278cfbbdacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.952619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92770892-2d55-402e-903d-3278cfbbdacc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.952897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92770892-2d55-402e-903d-3278cfbbdacc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.953168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92770892-2d55-402e-903d-3278cfbbdacc None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.953941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92770892-2d55-402e-903d-3278cfbbdacc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:08.954245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92770892-2d55-402e-903d-3278cfbbdacc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:08.955727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92770892-2d55-402e-903d-3278cfbbdacc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:08.967610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8347aa-9323-4d27-9558-0545e89a9303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:08.968275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8347aa-9323-4d27-9558-0545e89a9303 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.968566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8347aa-9323-4d27-9558-0545e89a9303 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.968849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8347aa-9323-4d27-9558-0545e89a9303 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.969593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8347aa-9323-4d27-9558-0545e89a9303 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:08.969911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8347aa-9323-4d27-9558-0545e89a9303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:08.971301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8347aa-9323-4d27-9558-0545e89a9303 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:08.973082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 289/569] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:08.974303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba83e5-0675-4fc0-816f-71f55c77d3ec None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:08.974629 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba83e5-0675-4fc0-816f-71f55c77d3ec None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:08.974907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba83e5-0675-4fc0-816f-71f55c77d3ec None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:08.976475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 290/570] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:08.977110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba83e5-0675-4fc0-816f-71f55c77d3ec None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:08.987306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 281/571] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:08.987306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8347aa-9323-4d27-9558-0545e89a9303 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.034435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:09.038966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:09.079363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba83e5-0675-4fc0-816f-71f55c77d3ec None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.084324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8347aa-9323-4d27-9558-0545e89a9303 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.112429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e168cd-2c03-4132-8cb7-c488155ed30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.113085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e168cd-2c03-4132-8cb7-c488155ed30b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.113373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e168cd-2c03-4132-8cb7-c488155ed30b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.113651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e168cd-2c03-4132-8cb7-c488155ed30b None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.114389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e168cd-2c03-4132-8cb7-c488155ed30b None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:09.114726 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e168cd-2c03-4132-8cb7-c488155ed30b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:09.116108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e168cd-2c03-4132-8cb7-c488155ed30b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:09.117387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d6f8c-1a00-44d1-af41-6186a31f92c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.118632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d6f8c-1a00-44d1-af41-6186a31f92c9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.119218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d6f8c-1a00-44d1-af41-6186a31f92c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.119505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d6f8c-1a00-44d1-af41-6186a31f92c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.120214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d6f8c-1a00-44d1-af41-6186a31f92c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:09.120524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d6f8c-1a00-44d1-af41-6186a31f92c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:09.122295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d6f8c-1a00-44d1-af41-6186a31f92c9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:09.124514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 291/572] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:09.125291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e168cd-2c03-4132-8cb7-c488155ed30b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.135445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 282/573] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:09.136938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77d560-179e-4ae3-a535-4a88f3a6f8bd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.137223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77d560-179e-4ae3-a535-4a88f3a6f8bd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.137496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77d560-179e-4ae3-a535-4a88f3a6f8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.153323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-fd77d560-179e-4ae3-a535-4a88f3a6f8bd None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:09.154521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fd77d560-179e-4ae3-a535-4a88f3a6f8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:09.219242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 283/574] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:09.220046 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77d560-179e-4ae3-a535-4a88f3a6f8bd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.223156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e168cd-2c03-4132-8cb7-c488155ed30b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.239920 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:09.258047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942fcf8-52d9-4689-8c16-8ea2e5b84cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.259251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942fcf8-52d9-4689-8c16-8ea2e5b84cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.259536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942fcf8-52d9-4689-8c16-8ea2e5b84cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.259829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942fcf8-52d9-4689-8c16-8ea2e5b84cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.260628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942fcf8-52d9-4689-8c16-8ea2e5b84cb6 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=1285406084604bd19287de9e1beeeaa8) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:09.261196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942fcf8-52d9-4689-8c16-8ea2e5b84cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:09.262657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942fcf8-52d9-4689-8c16-8ea2e5b84cb6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:09.266402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:09.268292 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 292/575] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/role_assignments?user.id=1285406084604bd19287de9e1beeeaa8&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:09.269207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942fcf8-52d9-4689-8c16-8ea2e5b84cb6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.326953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77d560-179e-4ae3-a535-4a88f3a6f8bd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.363545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5f086-f6ad-425f-b10d-2b1003d97b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.368553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5f086-f6ad-425f-b10d-2b1003d97b56 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.368553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5f086-f6ad-425f-b10d-2b1003d97b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.368553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5f086-f6ad-425f-b10d-2b1003d97b56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.379802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5f086-f6ad-425f-b10d-2b1003d97b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:09.381277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5f086-f6ad-425f-b10d-2b1003d97b56 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:09.397135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942fcf8-52d9-4689-8c16-8ea2e5b84cb6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f7f5f086-f6ad-425f-b10d-2b1003d97b56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:09.400424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:09.401513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:09.402495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:09.402495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:09.402495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:09.402495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:09.402495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 284/576] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:09.402830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5f086-f6ad-425f-b10d-2b1003d97b56 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.437286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b8de7-2fe6-48ed-b4e9-ff69f499ef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.438415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b8de7-2fe6-48ed-b4e9-ff69f499ef4e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.438886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b8de7-2fe6-48ed-b4e9-ff69f499ef4e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.439160 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b8de7-2fe6-48ed-b4e9-ff69f499ef4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.451522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b8de7-2fe6-48ed-b4e9-ff69f499ef4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:09.453018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b8de7-2fe6-48ed-b4e9-ff69f499ef4e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f92b8de7-2fe6-48ed-b4e9-ff69f499ef4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:09.473122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:09.474042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:09.475005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:09.475005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:09.475005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:09.475005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:09.475005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 293/577] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:09.475275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b8de7-2fe6-48ed-b4e9-ff69f499ef4e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.508212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:09.510055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:09.517897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5f086-f6ad-425f-b10d-2b1003d97b56 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.553761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd6eb-cc2e-4fe2-8738-ef129eac8d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.554590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd6eb-cc2e-4fe2-8738-ef129eac8d82 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.554878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd6eb-cc2e-4fe2-8738-ef129eac8d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.555691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fd6eb-cc2e-4fe2-8738-ef129eac8d82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.563205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fd6eb-cc2e-4fe2-8738-ef129eac8d82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:09.564553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fd6eb-cc2e-4fe2-8738-ef129eac8d82 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-364fd6eb-cc2e-4fe2-8738-ef129eac8d82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:09.578737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:09.579623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:09.580507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:09.580507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:09.580507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:09.580507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:09.580507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:09.580507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:09.580507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:09.580507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:09.580507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 285/578] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:09.581000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd6eb-cc2e-4fe2-8738-ef129eac8d82 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.590063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b8de7-2fe6-48ed-b4e9-ff69f499ef4e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.625130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73461b67-7d40-44c4-b95a-e0072aba10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.626047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73461b67-7d40-44c4-b95a-e0072aba10c3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.626331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73461b67-7d40-44c4-b95a-e0072aba10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.626636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73461b67-7d40-44c4-b95a-e0072aba10c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.637174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73461b67-7d40-44c4-b95a-e0072aba10c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-73461b67-7d40-44c4-b95a-e0072aba10c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:09.644842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:09.645785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:09.646681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 294/579] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:09.647385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73461b67-7d40-44c4-b95a-e0072aba10c3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.696264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fd6eb-cc2e-4fe2-8738-ef129eac8d82 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.733870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5940a-64e5-40fe-8c97-5bf9200624fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.734584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5940a-64e5-40fe-8c97-5bf9200624fa None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.734878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5940a-64e5-40fe-8c97-5bf9200624fa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.735162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5940a-64e5-40fe-8c97-5bf9200624fa None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.735886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5940a-64e5-40fe-8c97-5bf9200624fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:09.736192 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5940a-64e5-40fe-8c97-5bf9200624fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:09.737573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f5940a-64e5-40fe-8c97-5bf9200624fa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:09.746185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:09.749833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 286/580] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:09.750820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5940a-64e5-40fe-8c97-5bf9200624fa None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.764616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73461b67-7d40-44c4-b95a-e0072aba10c3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.770329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:09.799188 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b712e-31dc-46e1-8f6c-3bfa0e1df142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.799862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b712e-31dc-46e1-8f6c-3bfa0e1df142 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.800153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b712e-31dc-46e1-8f6c-3bfa0e1df142 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.800428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b712e-31dc-46e1-8f6c-3bfa0e1df142 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.801155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b712e-31dc-46e1-8f6c-3bfa0e1df142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:09.801468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b712e-31dc-46e1-8f6c-3bfa0e1df142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:09.802913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b712e-31dc-46e1-8f6c-3bfa0e1df142 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:09.819021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 295/581] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:09.820259 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b712e-31dc-46e1-8f6c-3bfa0e1df142 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.874284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5940a-64e5-40fe-8c97-5bf9200624fa None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.908835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a853b5-3581-4d82-a9a1-7b62370d4695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.909581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a853b5-3581-4d82-a9a1-7b62370d4695 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.909871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a853b5-3581-4d82-a9a1-7b62370d4695 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.910150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a853b5-3581-4d82-a9a1-7b62370d4695 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.910989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a853b5-3581-4d82-a9a1-7b62370d4695 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:09.911302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a853b5-3581-4d82-a9a1-7b62370d4695 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:09.912739 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a853b5-3581-4d82-a9a1-7b62370d4695 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:09.927485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 287/582] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:09.927801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b712e-31dc-46e1-8f6c-3bfa0e1df142 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:09.928459 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a853b5-3581-4d82-a9a1-7b62370d4695 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:09.971354 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94207d-b5d7-4daa-a5f0-5096dba2f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:09.972481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94207d-b5d7-4daa-a5f0-5096dba2f657 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:09.972942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94207d-b5d7-4daa-a5f0-5096dba2f657 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:09.973390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94207d-b5d7-4daa-a5f0-5096dba2f657 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:09.974424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94207d-b5d7-4daa-a5f0-5096dba2f657 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:09.974938 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94207d-b5d7-4daa-a5f0-5096dba2f657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:09.976605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94207d-b5d7-4daa-a5f0-5096dba2f657 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:09.986202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 296/583] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:09.987340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94207d-b5d7-4daa-a5f0-5096dba2f657 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.003454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:10.015466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:10.040505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a853b5-3581-4d82-a9a1-7b62370d4695 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:10.074227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7612feb-c3c3-4287-bce4-9ef0c2c57850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:10.075065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7612feb-c3c3-4287-bce4-9ef0c2c57850 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:10.075356 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7612feb-c3c3-4287-bce4-9ef0c2c57850 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:10.075696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7612feb-c3c3-4287-bce4-9ef0c2c57850 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:10.082791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7612feb-c3c3-4287-bce4-9ef0c2c57850 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:10.084244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7612feb-c3c3-4287-bce4-9ef0c2c57850 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:10.094209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94207d-b5d7-4daa-a5f0-5096dba2f657 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c7612feb-c3c3-4287-bce4-9ef0c2c57850 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:10.099172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:10.100205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:10.101314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 288/584] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:10.102586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7612feb-c3c3-4287-bce4-9ef0c2c57850 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.133777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7daeb3-20ea-4bf5-b938-75e27daf86cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:10.134984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7daeb3-20ea-4bf5-b938-75e27daf86cb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:10.135411 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7daeb3-20ea-4bf5-b938-75e27daf86cb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:10.135829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7daeb3-20ea-4bf5-b938-75e27daf86cb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:10.144503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7daeb3-20ea-4bf5-b938-75e27daf86cb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:10.146279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7daeb3-20ea-4bf5-b938-75e27daf86cb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1d7daeb3-20ea-4bf5-b938-75e27daf86cb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:10.161779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:10.162934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:10.164096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:10.164096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:10.164096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:10.164096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:10.164096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:10.164096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:10.164096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:10.164096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:10.164096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:51:10.164096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:10.167285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 297/585] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:10.168361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7daeb3-20ea-4bf5-b938-75e27daf86cb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.209230 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7612feb-c3c3-4287-bce4-9ef0c2c57850 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:10.242768 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f037e1-f4a8-44f2-8a69-3548e33f43c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:10.243655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f037e1-f4a8-44f2-8a69-3548e33f43c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:10.243942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f037e1-f4a8-44f2-8a69-3548e33f43c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:10.244224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f037e1-f4a8-44f2-8a69-3548e33f43c9 None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/490d773bf21446789b77c5793a518a5a/roles/4e24596391d3466eacb50a9e923e07ab` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:10.246162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:10.247606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:10.259640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f037e1-f4a8-44f2-8a69-3548e33f43c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=490d773bf21446789b77c5793a518a5a, role_id=4e24596391d3466eacb50a9e923e07ab)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:10.261353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f037e1-f4a8-44f2-8a69-3548e33f43c9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:10.276230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7daeb3-20ea-4bf5-b938-75e27daf86cb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:10.286842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 289/586] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:51:10 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/490d773bf21446789b77c5793a518a5a/roles/4e24596391d3466eacb50a9e923e07ab => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:51:10.286842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f037e1-f4a8-44f2-8a69-3548e33f43c9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.313843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781c45-b306-4f23-9409-c38cdf4099e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:10.314764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781c45-b306-4f23-9409-c38cdf4099e1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:10.315056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781c45-b306-4f23-9409-c38cdf4099e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:10.315336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea781c45-b306-4f23-9409-c38cdf4099e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:10.330229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea781c45-b306-4f23-9409-c38cdf4099e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:10.331884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea781c45-b306-4f23-9409-c38cdf4099e1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ea781c45-b306-4f23-9409-c38cdf4099e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:10.351631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:10.353311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:10.354232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:10.354232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:10.354232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:10.354232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:10.354232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 298/587] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:10.354556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781c45-b306-4f23-9409-c38cdf4099e1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.427965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f037e1-f4a8-44f2-8a69-3548e33f43c9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:10.467227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45f2bf-104e-47c9-aa75-38cd9509fd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:10.468347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45f2bf-104e-47c9-aa75-38cd9509fd54 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:10.468839 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45f2bf-104e-47c9-aa75-38cd9509fd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:10.469323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45f2bf-104e-47c9-aa75-38cd9509fd54 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:10.470510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45f2bf-104e-47c9-aa75-38cd9509fd54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:10.471065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45f2bf-104e-47c9-aa75-38cd9509fd54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:10.471649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea781c45-b306-4f23-9409-c38cdf4099e1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:10.473319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45f2bf-104e-47c9-aa75-38cd9509fd54 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:10.496147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 290/588] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 26 20:51:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:10.497595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45f2bf-104e-47c9-aa75-38cd9509fd54 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.505758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:10.518252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7ac04-c3fc-4f73-a79b-a6e3f93aa1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:10.518970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7ac04-c3fc-4f73-a79b-a6e3f93aa1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:10.519259 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7ac04-c3fc-4f73-a79b-a6e3f93aa1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:10.519545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7ac04-c3fc-4f73-a79b-a6e3f93aa1f6 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:10.520392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7ac04-c3fc-4f73-a79b-a6e3f93aa1f6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:10.520711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7ac04-c3fc-4f73-a79b-a6e3f93aa1f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:10.522159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7ac04-c3fc-4f73-a79b-a6e3f93aa1f6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:10.542054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 299/589] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 26 20:51:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:10.542737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7ac04-c3fc-4f73-a79b-a6e3f93aa1f6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.556019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:10.620445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45f2bf-104e-47c9-aa75-38cd9509fd54 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:10.646027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7ac04-c3fc-4f73-a79b-a6e3f93aa1f6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:10.655862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b8447-76d2-4ca9-b99e-13923c16585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:10.656537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b8447-76d2-4ca9-b99e-13923c16585c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:10.656823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b8447-76d2-4ca9-b99e-13923c16585c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:10.657103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b8447-76d2-4ca9-b99e-13923c16585c None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:10.657832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b8447-76d2-4ca9-b99e-13923c16585c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:10.658141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b8447-76d2-4ca9-b99e-13923c16585c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:10.659554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b8447-76d2-4ca9-b99e-13923c16585c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:10.672140 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 291/590] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:10.672965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b8447-76d2-4ca9-b99e-13923c16585c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.682432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbd207-90e2-4b9a-a112-cec482b2220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:10.683278 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbd207-90e2-4b9a-a112-cec482b2220a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:10.683792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbd207-90e2-4b9a-a112-cec482b2220a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:10.684170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbd207-90e2-4b9a-a112-cec482b2220a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:10.694597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbd207-90e2-4b9a-a112-cec482b2220a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:10.696301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbd207-90e2-4b9a-a112-cec482b2220a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-56dbd207-90e2-4b9a-a112-cec482b2220a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:10.714706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:10.715894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:10.716816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:10.716816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:10.716816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:10.716816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:10.721737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 300/591] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:10.722690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbd207-90e2-4b9a-a112-cec482b2220a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.758372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:10.783005 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b8447-76d2-4ca9-b99e-13923c16585c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:10.798882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:10.823520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bd7ee-45dd-4b6a-a318-77d87cf0c34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:10.824456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bd7ee-45dd-4b6a-a318-77d87cf0c34f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:10.824890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bd7ee-45dd-4b6a-a318-77d87cf0c34f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:10.825246 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bd7ee-45dd-4b6a-a318-77d87cf0c34f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:10.836676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbd207-90e2-4b9a-a112-cec482b2220a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:10.838031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bd7ee-45dd-4b6a-a318-77d87cf0c34f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:10.840125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bd7ee-45dd-4b6a-a318-77d87cf0c34f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5a4bd7ee-45dd-4b6a-a318-77d87cf0c34f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:10.859032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:10.860011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:10.860891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:10.860891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:10.860891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:10.860891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:10.865098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 292/592] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:10.866126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bd7ee-45dd-4b6a-a318-77d87cf0c34f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.881054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c9d3e-1f9b-429d-905b-d0c95898ad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:10.882272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9d3e-1f9b-429d-905b-d0c95898ad60 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:10.882616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9d3e-1f9b-429d-905b-d0c95898ad60 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:10.882904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9d3e-1f9b-429d-905b-d0c95898ad60 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:10.893146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c9d3e-1f9b-429d-905b-d0c95898ad60 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:10.895181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c9d3e-1f9b-429d-905b-d0c95898ad60 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f60c9d3e-1f9b-429d-905b-d0c95898ad60 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:10.916305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:10.917321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:10.918272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 301/593] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:10.918895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c9d3e-1f9b-429d-905b-d0c95898ad60 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:10.991556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bd7ee-45dd-4b6a-a318-77d87cf0c34f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.028249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61452dd-5b38-44bf-8e76-5b6d1a893e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.028982 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61452dd-5b38-44bf-8e76-5b6d1a893e87 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.029269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61452dd-5b38-44bf-8e76-5b6d1a893e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.029602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61452dd-5b38-44bf-8e76-5b6d1a893e87 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.030333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61452dd-5b38-44bf-8e76-5b6d1a893e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:11.030728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61452dd-5b38-44bf-8e76-5b6d1a893e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:11.032201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61452dd-5b38-44bf-8e76-5b6d1a893e87 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:11.038813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c9d3e-1f9b-429d-905b-d0c95898ad60 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.049271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 293/594] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:11.049271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61452dd-5b38-44bf-8e76-5b6d1a893e87 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:11.058825 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:11.064710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:11.082362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a900fa7-9edd-4285-b955-b7cdeed9a5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.083379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a900fa7-9edd-4285-b955-b7cdeed9a5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.083960 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a900fa7-9edd-4285-b955-b7cdeed9a5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.084463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a900fa7-9edd-4285-b955-b7cdeed9a5c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.085434 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a900fa7-9edd-4285-b955-b7cdeed9a5c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:11.085970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a900fa7-9edd-4285-b955-b7cdeed9a5c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:11.088174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a900fa7-9edd-4285-b955-b7cdeed9a5c8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:11.105498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 302/595] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:11.106878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a900fa7-9edd-4285-b955-b7cdeed9a5c8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:11.156441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61452dd-5b38-44bf-8e76-5b6d1a893e87 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.191958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445c861-aba7-4825-b092-51a717f9e3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.192561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445c861-aba7-4825-b092-51a717f9e3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.192907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445c861-aba7-4825-b092-51a717f9e3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.193237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445c861-aba7-4825-b092-51a717f9e3a6 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.194079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445c861-aba7-4825-b092-51a717f9e3a6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:11.194393 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445c861-aba7-4825-b092-51a717f9e3a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:11.196492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445c861-aba7-4825-b092-51a717f9e3a6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:11.215312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 294/596] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 26 20:51:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:11.216217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445c861-aba7-4825-b092-51a717f9e3a6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:11.217269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a900fa7-9edd-4285-b955-b7cdeed9a5c8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.253588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1f16a-a5dd-4081-9820-cd66a44738a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.254845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f16a-a5dd-4081-9820-cd66a44738a6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.255368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f16a-a5dd-4081-9820-cd66a44738a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.255848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1f16a-a5dd-4081-9820-cd66a44738a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.265030 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1f16a-a5dd-4081-9820-cd66a44738a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:11.267213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1f16a-a5dd-4081-9820-cd66a44738a6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-efa1f16a-a5dd-4081-9820-cd66a44738a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:11.284592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:11.285555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:11.286496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:11.286496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:11.286496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:11.286496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:11.286496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:11.286496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:11.286496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:11.286496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:11.293759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 303/597] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:11.295115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1f16a-a5dd-4081-9820-cd66a44738a6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:11.299886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:11.326826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:11.327005 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445c861-aba7-4825-b092-51a717f9e3a6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.361975 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7390a09-2627-4c20-ae78-589d4b8ae7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.362835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390a09-2627-4c20-ae78-589d4b8ae7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.363121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390a09-2627-4c20-ae78-589d4b8ae7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.363400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7390a09-2627-4c20-ae78-589d4b8ae7b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.371035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7390a09-2627-4c20-ae78-589d4b8ae7b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:11.372487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7390a09-2627-4c20-ae78-589d4b8ae7b8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a7390a09-2627-4c20-ae78-589d4b8ae7b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:11.387333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:11.388300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:11.389193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:11.389193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:11.389193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:11.389193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:11.389193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:11.389193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:11.389193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:11.389193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:11.393346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 295/598] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:11.394195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7390a09-2627-4c20-ae78-589d4b8ae7b8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:11.408744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1f16a-a5dd-4081-9820-cd66a44738a6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.446707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fb33e-c166-4e8c-9ed3-7d53027a0e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.447972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fb33e-c166-4e8c-9ed3-7d53027a0e63 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.448457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fb33e-c166-4e8c-9ed3-7d53027a0e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.448958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fb33e-c166-4e8c-9ed3-7d53027a0e63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.461557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fb33e-c166-4e8c-9ed3-7d53027a0e63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:11.463553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fb33e-c166-4e8c-9ed3-7d53027a0e63 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-817fb33e-c166-4e8c-9ed3-7d53027a0e63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:11.481344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:11.482786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:11.483723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:11.483723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:11.483723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:11.483723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:11.483723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 304/599] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:11.483723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fb33e-c166-4e8c-9ed3-7d53027a0e63 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:11.509275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7390a09-2627-4c20-ae78-589d4b8ae7b8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.544722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73fc23-0e41-40fb-888f-3ac6e91c802f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.545394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73fc23-0e41-40fb-888f-3ac6e91c802f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.545679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73fc23-0e41-40fb-888f-3ac6e91c802f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.545964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73fc23-0e41-40fb-888f-3ac6e91c802f None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.546786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73fc23-0e41-40fb-888f-3ac6e91c802f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:11.547102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73fc23-0e41-40fb-888f-3ac6e91c802f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:11.548538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73fc23-0e41-40fb-888f-3ac6e91c802f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:11.550620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:11.563624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:11.563898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 296/600] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:11.565263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca2e9d-584f-4c46-bdbb-e9dfca3fde99 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.565554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca2e9d-584f-4c46-bdbb-e9dfca3fde99 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.565830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca2e9d-584f-4c46-bdbb-e9dfca3fde99 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.570637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 297/601] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:11.570637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca2e9d-584f-4c46-bdbb-e9dfca3fde99 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:11.606040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fb33e-c166-4e8c-9ed3-7d53027a0e63 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.643535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec74ae-1746-4786-8cb5-fd8c66c51d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.644196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec74ae-1746-4786-8cb5-fd8c66c51d95 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.644480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec74ae-1746-4786-8cb5-fd8c66c51d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.644756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec74ae-1746-4786-8cb5-fd8c66c51d95 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.645587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec74ae-1746-4786-8cb5-fd8c66c51d95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:11.645905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec74ae-1746-4786-8cb5-fd8c66c51d95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:11.647579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec74ae-1746-4786-8cb5-fd8c66c51d95 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:11.660325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 305/602] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:11.661179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec74ae-1746-4786-8cb5-fd8c66c51d95 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:11.676681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca2e9d-584f-4c46-bdbb-e9dfca3fde99 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.711613 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dfc36f-c51c-4fdb-9266-5fef5dd20a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.712267 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfc36f-c51c-4fdb-9266-5fef5dd20a7e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.712551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfc36f-c51c-4fdb-9266-5fef5dd20a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.712822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dfc36f-c51c-4fdb-9266-5fef5dd20a7e None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.713537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfc36f-c51c-4fdb-9266-5fef5dd20a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:11.713843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfc36f-c51c-4fdb-9266-5fef5dd20a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:11.715224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dfc36f-c51c-4fdb-9266-5fef5dd20a7e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:11.726058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 298/603] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:11.727790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd082e7-166d-485b-8b39-03f7c7eb0720 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.728077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd082e7-166d-485b-8b39-03f7c7eb0720 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.728354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd082e7-166d-485b-8b39-03f7c7eb0720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.744620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-afd082e7-166d-485b-8b39-03f7c7eb0720 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:11.745793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-afd082e7-166d-485b-8b39-03f7c7eb0720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:11.771297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec74ae-1746-4786-8cb5-fd8c66c51d95 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.794577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:11.800121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:11.814818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6e377-1940-4d7e-a4e1-edc97453ee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.815478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6e377-1940-4d7e-a4e1-edc97453ee5f None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.815768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6e377-1940-4d7e-a4e1-edc97453ee5f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.816053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6e377-1940-4d7e-a4e1-edc97453ee5f None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/2c3bd8dd462846bf9423f8528aa5c489/roles/4e24596391d3466eacb50a9e923e07ab` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:11.826529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 299/604] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:11.826529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd082e7-166d-485b-8b39-03f7c7eb0720 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:11.845037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6e377-1940-4d7e-a4e1-edc97453ee5f None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=2c3bd8dd462846bf9423f8528aa5c489, role_id=4e24596391d3466eacb50a9e923e07ab)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:11.846539 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6e377-1940-4d7e-a4e1-edc97453ee5f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:11.859448 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 306/605] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:51:11 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/2c3bd8dd462846bf9423f8528aa5c489/roles/4e24596391d3466eacb50a9e923e07ab => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:51:11.860237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6e377-1940-4d7e-a4e1-edc97453ee5f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:11.953308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd082e7-166d-485b-8b39-03f7c7eb0720 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.995688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6e377-1940-4d7e-a4e1-edc97453ee5f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:11.998130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f43acd-8ebf-4d53-ad9c-580722d37c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:11.999091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f43acd-8ebf-4d53-ad9c-580722d37c11 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:11.999381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f43acd-8ebf-4d53-ad9c-580722d37c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:11.999660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f43acd-8ebf-4d53-ad9c-580722d37c11 None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/afe351334485400a8c11f17492c9ff5a/roles/4e24596391d3466eacb50a9e923e07ab` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:12.017125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f43acd-8ebf-4d53-ad9c-580722d37c11 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=afe351334485400a8c11f17492c9ff5a, role_id=4e24596391d3466eacb50a9e923e07ab)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:12.018649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f43acd-8ebf-4d53-ad9c-580722d37c11 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:12.041341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 300/606] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:51:11 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/afe351334485400a8c11f17492c9ff5a/roles/4e24596391d3466eacb50a9e923e07ab => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:51:12.041341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f43acd-8ebf-4d53-ad9c-580722d37c11 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:12.054251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378355d-ae20-4c82-90b3-89148c7275c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:12.055746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378355d-ae20-4c82-90b3-89148c7275c5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:12.056095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378355d-ae20-4c82-90b3-89148c7275c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:12.056376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378355d-ae20-4c82-90b3-89148c7275c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:12.077833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:12.079741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378355d-ae20-4c82-90b3-89148c7275c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:12.081240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378355d-ae20-4c82-90b3-89148c7275c5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c378355d-ae20-4c82-90b3-89148c7275c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:12.106374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:12.107426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:12.108331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:12.108331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:12.108331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:12.108331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:12.108331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:12.108331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:12.108331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:12.108331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:12.108331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 307/607] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:12.115507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378355d-ae20-4c82-90b3-89148c7275c5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:12.162347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f43acd-8ebf-4d53-ad9c-580722d37c11 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:12.200550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:12.243055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed56057-85af-4000-88af-bb40427dac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:12.243935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed56057-85af-4000-88af-bb40427dac59 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:12.244225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed56057-85af-4000-88af-bb40427dac59 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:12.244502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed56057-85af-4000-88af-bb40427dac59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:12.270487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed56057-85af-4000-88af-bb40427dac59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8ed56057-85af-4000-88af-bb40427dac59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:12.280833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:12.281849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:12.283180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 301/608] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:12.292061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed56057-85af-4000-88af-bb40427dac59 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:12.316175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378355d-ae20-4c82-90b3-89148c7275c5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:12.362256 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca5918-826e-45a6-b7e8-5b592117cc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:12.363064 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5918-826e-45a6-b7e8-5b592117cc3e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:12.363352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5918-826e-45a6-b7e8-5b592117cc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:12.363630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5918-826e-45a6-b7e8-5b592117cc3e None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:12.364478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca5918-826e-45a6-b7e8-5b592117cc3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:12.364786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca5918-826e-45a6-b7e8-5b592117cc3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:12.366474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca5918-826e-45a6-b7e8-5b592117cc3e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:12.392378 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 308/609] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:12.397363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:12.398707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca5918-826e-45a6-b7e8-5b592117cc3e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:12.400455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed56057-85af-4000-88af-bb40427dac59 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:12.445399 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20409f8c-8298-4d3f-938d-44b608f76b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:12.446734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20409f8c-8298-4d3f-938d-44b608f76b92 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:12.450482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20409f8c-8298-4d3f-938d-44b608f76b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:12.450482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20409f8c-8298-4d3f-938d-44b608f76b92 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:12.450482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20409f8c-8298-4d3f-938d-44b608f76b92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:12.450482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20409f8c-8298-4d3f-938d-44b608f76b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:12.450482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20409f8c-8298-4d3f-938d-44b608f76b92 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:12.476100 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 302/610] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:12.480873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20409f8c-8298-4d3f-938d-44b608f76b92 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:12.584003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:12.588775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca5918-826e-45a6-b7e8-5b592117cc3e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:12.626084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1a96a-f7c6-48ee-902b-5be1dd977009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:12.627379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1a96a-f7c6-48ee-902b-5be1dd977009 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:12.627885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1a96a-f7c6-48ee-902b-5be1dd977009 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:12.628376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1a96a-f7c6-48ee-902b-5be1dd977009 None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/94b8d8b089264b5192a651b41bf2a205/roles/4e24596391d3466eacb50a9e923e07ab` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:12.644142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1a96a-f7c6-48ee-902b-5be1dd977009 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=94b8d8b089264b5192a651b41bf2a205, role_id=4e24596391d3466eacb50a9e923e07ab)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:12.645943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1a96a-f7c6-48ee-902b-5be1dd977009 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:12.663702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 309/611] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:51:12 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/94b8d8b089264b5192a651b41bf2a205/roles/4e24596391d3466eacb50a9e923e07ab => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:51:12.665964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11107f7-4087-45a6-b2e7-8a3f0ae40203 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:12.666530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11107f7-4087-45a6-b2e7-8a3f0ae40203 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:12.667031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11107f7-4087-45a6-b2e7-8a3f0ae40203 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:12.668993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 310/612] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:12.679792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20409f8c-8298-4d3f-938d-44b608f76b92 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:12.684884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bcd00-c160-4382-88ef-0fd7b0069703 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:12.685573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bcd00-c160-4382-88ef-0fd7b0069703 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:12.686098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bcd00-c160-4382-88ef-0fd7b0069703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:12.718114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c01bcd00-c160-4382-88ef-0fd7b0069703 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:12.719393 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c01bcd00-c160-4382-88ef-0fd7b0069703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:12.730562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd992-9e1b-4031-88b0-d89ce5130df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:12.731748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd992-9e1b-4031-88b0-d89ce5130df8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:12.732258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd992-9e1b-4031-88b0-d89ce5130df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:12.732757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4abd992-9e1b-4031-88b0-d89ce5130df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:12.746171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abd992-9e1b-4031-88b0-d89ce5130df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:12.748296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4abd992-9e1b-4031-88b0-d89ce5130df8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a4abd992-9e1b-4031-88b0-d89ce5130df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:12.768850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:12.769823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:12.770811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:12.770811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:12.770811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:12.770811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:12.773281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 303/613] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:12.778433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd992-9e1b-4031-88b0-d89ce5130df8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:12.800885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:12.876108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 311/614] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:12.902910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bcd00-c160-4382-88ef-0fd7b0069703 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:12.936896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4abd992-9e1b-4031-88b0-d89ce5130df8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:12.972284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a06bfb-321d-4d34-a70b-c36d6822c385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:12.973108 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a06bfb-321d-4d34-a70b-c36d6822c385 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:12.973484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a06bfb-321d-4d34-a70b-c36d6822c385 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:12.973878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a06bfb-321d-4d34-a70b-c36d6822c385 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:12.974734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a06bfb-321d-4d34-a70b-c36d6822c385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:12.975191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a06bfb-321d-4d34-a70b-c36d6822c385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:12.976865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a06bfb-321d-4d34-a70b-c36d6822c385 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:12.989503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 304/615] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:12.995009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a06bfb-321d-4d34-a70b-c36d6822c385 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:13.063815 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:13.101088 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bcd00-c160-4382-88ef-0fd7b0069703 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:13.118886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:13.171067 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b32e44-f667-4813-96d4-c2bf78df1b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:13.172236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b32e44-f667-4813-96d4-c2bf78df1b49 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:13.173686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b32e44-f667-4813-96d4-c2bf78df1b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:13.174198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b32e44-f667-4813-96d4-c2bf78df1b49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:13.177727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a06bfb-321d-4d34-a70b-c36d6822c385 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:13.199463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b32e44-f667-4813-96d4-c2bf78df1b49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-57b32e44-f667-4813-96d4-c2bf78df1b49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:13.211659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:13.212603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:13.213459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:13.224722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 312/616] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:13.242989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b32e44-f667-4813-96d4-c2bf78df1b49 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:13.259718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431122a-865a-43ed-a66d-ba39429990be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:13.261162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431122a-865a-43ed-a66d-ba39429990be None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:13.262926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431122a-865a-43ed-a66d-ba39429990be None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:13.263699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431122a-865a-43ed-a66d-ba39429990be None admin] PATH_INFO: `/v3/users/swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:13.280665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431122a-865a-43ed-a66d-ba39429990be None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:13.282432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431122a-865a-43ed-a66d-ba39429990be None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4431122a-865a-43ed-a66d-ba39429990be None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:13.318116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:13.319187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:13.320169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:13.320169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:13.320169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:13.320169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:13.320169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:13.320169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:13.320169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:13.320169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:13.320169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:13.320169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:13.324889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 305/617] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:13.330346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431122a-865a-43ed-a66d-ba39429990be None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:13.450534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b32e44-f667-4813-96d4-c2bf78df1b49 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:13.463353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431122a-865a-43ed-a66d-ba39429990be None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:13.514889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:13.517964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b9518-d517-4299-816a-05d8466c1399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:13.518679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b9518-d517-4299-816a-05d8466c1399 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:13.518963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b9518-d517-4299-816a-05d8466c1399 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:13.519244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b9518-d517-4299-816a-05d8466c1399 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:13.519998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b9518-d517-4299-816a-05d8466c1399 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:13.520303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b9518-d517-4299-816a-05d8466c1399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:13.521693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b9518-d517-4299-816a-05d8466c1399 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:13.524718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee2cc5-865c-4ad2-9895-52b77ef8c8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:13.525385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2cc5-865c-4ad2-9895-52b77ef8c8aa None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:13.525666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2cc5-865c-4ad2-9895-52b77ef8c8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:13.526009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2cc5-865c-4ad2-9895-52b77ef8c8aa None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:13.527419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:13.533194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 313/618] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:13.534536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee2cc5-865c-4ad2-9895-52b77ef8c8aa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:13.534870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee2cc5-865c-4ad2-9895-52b77ef8c8aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:13.536329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee2cc5-865c-4ad2-9895-52b77ef8c8aa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:13.538087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b9518-d517-4299-816a-05d8466c1399 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:13.577101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 306/619] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 26 20:51:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:13.582438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee2cc5-865c-4ad2-9895-52b77ef8c8aa None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:13.648075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b9518-d517-4299-816a-05d8466c1399 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:13.686757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30311a09-69e3-4a23-9f96-e0e596d1ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:13.687964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30311a09-69e3-4a23-9f96-e0e596d1ccf0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:13.688454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30311a09-69e3-4a23-9f96-e0e596d1ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:13.688923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30311a09-69e3-4a23-9f96-e0e596d1ccf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:13.700653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30311a09-69e3-4a23-9f96-e0e596d1ccf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:13.702390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30311a09-69e3-4a23-9f96-e0e596d1ccf0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-30311a09-69e3-4a23-9f96-e0e596d1ccf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:13.721662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:13.722945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:13.723965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:13.723965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:13.723965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:13.723965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:13.726035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 314/620] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:13.739478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30311a09-69e3-4a23-9f96-e0e596d1ccf0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:13.787591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee2cc5-865c-4ad2-9895-52b77ef8c8aa None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:13.851028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e76b-23de-4512-8f47-5ad390fbb0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:13.851866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e76b-23de-4512-8f47-5ad390fbb0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:13.852154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e76b-23de-4512-8f47-5ad390fbb0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:13.852470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e76b-23de-4512-8f47-5ad390fbb0b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:13.863045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:13.879797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4e76b-23de-4512-8f47-5ad390fbb0b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:13.881282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4e76b-23de-4512-8f47-5ad390fbb0b9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-67f4e76b-23de-4512-8f47-5ad390fbb0b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:13.915171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:13.916131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:13.917019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:13.917019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:13.917019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:13.917019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:13.917948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 307/621] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:13.921602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e76b-23de-4512-8f47-5ad390fbb0b9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:13.964502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30311a09-69e3-4a23-9f96-e0e596d1ccf0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:14.002876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:14.044513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0b68a-14e4-4ff6-a7bf-f4efe8344d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:14.045303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0b68a-14e4-4ff6-a7bf-f4efe8344d15 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.045588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0b68a-14e4-4ff6-a7bf-f4efe8344d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.045864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0b68a-14e4-4ff6-a7bf-f4efe8344d15 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.046630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0b68a-14e4-4ff6-a7bf-f4efe8344d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:14.046940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0b68a-14e4-4ff6-a7bf-f4efe8344d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:14.048337 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0b68a-14e4-4ff6-a7bf-f4efe8344d15 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:14.060332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e76b-23de-4512-8f47-5ad390fbb0b9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:14.069022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 315/622] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:14.075585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0b68a-14e4-4ff6-a7bf-f4efe8344d15 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:14.097898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346572e3-450a-421f-9946-0f65a6c987b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:14.098710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346572e3-450a-421f-9946-0f65a6c987b3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.099006 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346572e3-450a-421f-9946-0f65a6c987b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.099270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346572e3-450a-421f-9946-0f65a6c987b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.100011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346572e3-450a-421f-9946-0f65a6c987b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:14.100326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346572e3-450a-421f-9946-0f65a6c987b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:14.102713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346572e3-450a-421f-9946-0f65a6c987b3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:14.115878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 308/623] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:14.121436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346572e3-450a-421f-9946-0f65a6c987b3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:14.219731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0b68a-14e4-4ff6-a7bf-f4efe8344d15 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:14.232858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:14.237654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346572e3-450a-421f-9946-0f65a6c987b3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:14.250797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:14.254719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c79ef-8d01-49f6-a722-22e9a4d02860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:14.255285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c79ef-8d01-49f6-a722-22e9a4d02860 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.255568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c79ef-8d01-49f6-a722-22e9a4d02860 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.255843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c79ef-8d01-49f6-a722-22e9a4d02860 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.265243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c79ef-8d01-49f6-a722-22e9a4d02860 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:14.267154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c79ef-8d01-49f6-a722-22e9a4d02860 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:14.280273 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4bd12-eab1-453c-b693-6255f97d832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:14.281385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4bd12-eab1-453c-b693-6255f97d832c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.281873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4bd12-eab1-453c-b693-6255f97d832c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.282350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4bd12-eab1-453c-b693-6255f97d832c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5a6c79ef-8d01-49f6-a722-22e9a4d02860 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:14.282564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:14.283527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:14.285201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 316/624] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 26 20:51:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:14.289645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c79ef-8d01-49f6-a722-22e9a4d02860 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:14.294500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4bd12-eab1-453c-b693-6255f97d832c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:14.295349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed4bd12-eab1-453c-b693-6255f97d832c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1ed4bd12-eab1-453c-b693-6255f97d832c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:14.310860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:14.311861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:14.313194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:14.313194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:14.313194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:14.313194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:14.313194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:14.313194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:14.313194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:14.313194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:14.313194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 309/625] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:14.318214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4bd12-eab1-453c-b693-6255f97d832c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:14.424236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4bd12-eab1-453c-b693-6255f97d832c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:14.429243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c79ef-8d01-49f6-a722-22e9a4d02860 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:14.459958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc62304-483a-44f1-93d5-0982a5f618b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:14.460625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc62304-483a-44f1-93d5-0982a5f618b5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.460909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc62304-483a-44f1-93d5-0982a5f618b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.461705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc62304-483a-44f1-93d5-0982a5f618b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.462586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc62304-483a-44f1-93d5-0982a5f618b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:14.462901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc62304-483a-44f1-93d5-0982a5f618b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:14.464332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc62304-483a-44f1-93d5-0982a5f618b5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:14.475908 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f815866-b1ef-4feb-a0f3-253b3d689e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:14.477693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f815866-b1ef-4feb-a0f3-253b3d689e9d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.478181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f815866-b1ef-4feb-a0f3-253b3d689e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.478686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f815866-b1ef-4feb-a0f3-253b3d689e9d None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.479512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 310/626] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:14.480103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f815866-b1ef-4feb-a0f3-253b3d689e9d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:14.480674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f815866-b1ef-4feb-a0f3-253b3d689e9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:14.482313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f815866-b1ef-4feb-a0f3-253b3d689e9d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:14.485028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc62304-483a-44f1-93d5-0982a5f618b5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:14.502556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 317/627] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 26 20:51:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:14.508711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f815866-b1ef-4feb-a0f3-253b3d689e9d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:14.514549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:14.545198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:14.601688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc62304-483a-44f1-93d5-0982a5f618b5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:14.619905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f815866-b1ef-4feb-a0f3-253b3d689e9d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:14.635558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a154f61-d723-4082-9404-02aa59d3623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:14.636832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a154f61-d723-4082-9404-02aa59d3623e None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.637202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a154f61-d723-4082-9404-02aa59d3623e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.637544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a154f61-d723-4082-9404-02aa59d3623e None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/1285406084604bd19287de9e1beeeaa8/roles/4e24596391d3466eacb50a9e923e07ab` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.652676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a154f61-d723-4082-9404-02aa59d3623e None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=1285406084604bd19287de9e1beeeaa8, role_id=4e24596391d3466eacb50a9e923e07ab)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:14.654218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a154f61-d723-4082-9404-02aa59d3623e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:14.657401 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc2c599-fc64-48a6-a11a-8ff74946cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:14.658142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2c599-fc64-48a6-a11a-8ff74946cbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.658608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2c599-fc64-48a6-a11a-8ff74946cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.658959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc2c599-fc64-48a6-a11a-8ff74946cbb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.678957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 311/628] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:51:14 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/1285406084604bd19287de9e1beeeaa8/roles/4e24596391d3466eacb50a9e923e07ab => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:51:14.680804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9181cf2-603f-4094-802b-5f031801ed8b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.681151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9181cf2-603f-4094-802b-5f031801ed8b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.681434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9181cf2-603f-4094-802b-5f031801ed8b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.683030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 312/629] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:14.689673 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916c74-9c0f-4dc1-982f-6f73527766ae None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.690057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916c74-9c0f-4dc1-982f-6f73527766ae None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.690228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc916c74-9c0f-4dc1-982f-6f73527766ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.711706 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-fc916c74-9c0f-4dc1-982f-6f73527766ae None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:14.713030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fc916c74-9c0f-4dc1-982f-6f73527766ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:14.783730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:14.789809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 313/630] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:51:14.800393 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2c599-fc64-48a6-a11a-8ff74946cbb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:14.801972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc2c599-fc64-48a6-a11a-8ff74946cbb7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:14.824543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc916c74-9c0f-4dc1-982f-6f73527766ae None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:14.875151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:14.973523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 318/631] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:14.974912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71546d84-a08c-42aa-9178-d20f4e828cac None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:14.975202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71546d84-a08c-42aa-9178-d20f4e828cac None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:14.975481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71546d84-a08c-42aa-9178-d20f4e828cac None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:14.977053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 319/632] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:14.981778 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71546d84-a08c-42aa-9178-d20f4e828cac None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:14.984665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc916c74-9c0f-4dc1-982f-6f73527766ae None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:15.028933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb0e8e-e4bb-4a42-82cd-39c527c0a806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:15.030783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0e8e-e4bb-4a42-82cd-39c527c0a806 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.031255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0e8e-e4bb-4a42-82cd-39c527c0a806 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.031742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0e8e-e4bb-4a42-82cd-39c527c0a806 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.047327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb0e8e-e4bb-4a42-82cd-39c527c0a806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4ecb0e8e-e4bb-4a42-82cd-39c527c0a806 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:15.058355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:15.059362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:15.060638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 314/633] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:15.062695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5cc87-8432-4748-acae-feb04d1ec246 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.063205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5cc87-8432-4748-acae-feb04d1ec246 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.063681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5cc87-8432-4748-acae-feb04d1ec246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.085940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-01f5cc87-8432-4748-acae-feb04d1ec246 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:15.088825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-01f5cc87-8432-4748-acae-feb04d1ec246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:15.094378 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71546d84-a08c-42aa-9178-d20f4e828cac None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:15.127930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:15.135057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d99db3-ac44-473a-9e4e-c4f3b55e92e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:15.136065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d99db3-ac44-473a-9e4e-c4f3b55e92e5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.136361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d99db3-ac44-473a-9e4e-c4f3b55e92e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.136642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d99db3-ac44-473a-9e4e-c4f3b55e92e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.147782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d99db3-ac44-473a-9e4e-c4f3b55e92e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:15.149794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d99db3-ac44-473a-9e4e-c4f3b55e92e5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:15.165719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e0d99db3-ac44-473a-9e4e-c4f3b55e92e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:15.168693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:15.169652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:15.171019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:15.171019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:15.171019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:15.171019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:15.171019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 320/634] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:15.172258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d99db3-ac44-473a-9e4e-c4f3b55e92e5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:15.195033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 315/635] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:15.195033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5cc87-8432-4748-acae-feb04d1ec246 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:15.315958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5cc87-8432-4748-acae-feb04d1ec246 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:15.325878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d99db3-ac44-473a-9e4e-c4f3b55e92e5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:15.352381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92a8b3-4696-467f-bef9-54ffa5b83cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:15.353306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92a8b3-4696-467f-bef9-54ffa5b83cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.353306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92a8b3-4696-467f-bef9-54ffa5b83cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.353608 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92a8b3-4696-467f-bef9-54ffa5b83cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.354408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92a8b3-4696-467f-bef9-54ffa5b83cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:15.354755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92a8b3-4696-467f-bef9-54ffa5b83cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:15.356198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92a8b3-4696-467f-bef9-54ffa5b83cd8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:15.368396 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 316/636] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:15.369300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92a8b3-4696-467f-bef9-54ffa5b83cd8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:15.371491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee4f40-944e-4ecd-abdf-af1463d4369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:15.372765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee4f40-944e-4ecd-abdf-af1463d4369c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.373413 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee4f40-944e-4ecd-abdf-af1463d4369c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.374642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ee4f40-944e-4ecd-abdf-af1463d4369c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.375467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee4f40-944e-4ecd-abdf-af1463d4369c None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:15.375787 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee4f40-944e-4ecd-abdf-af1463d4369c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:15.377191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ee4f40-944e-4ecd-abdf-af1463d4369c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:15.386631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 321/637] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:15.387453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee4f40-944e-4ecd-abdf-af1463d4369c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:15.430231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:15.443440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:15.483267 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92a8b3-4696-467f-bef9-54ffa5b83cd8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:15.513653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ee4f40-944e-4ecd-abdf-af1463d4369c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:15.518668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bc773-f50f-4c38-bf3c-be264340aee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:15.519469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bc773-f50f-4c38-bf3c-be264340aee4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.519753 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bc773-f50f-4c38-bf3c-be264340aee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.520028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bc773-f50f-4c38-bf3c-be264340aee4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.531020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bc773-f50f-4c38-bf3c-be264340aee4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f45bc773-f50f-4c38-bf3c-be264340aee4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:15.538751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:15.539770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:15.540649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 317/638] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:15.541343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bc773-f50f-4c38-bf3c-be264340aee4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:15.555998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a715b1-9bf8-4368-b9ef-07683ecdf9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:15.556798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a715b1-9bf8-4368-b9ef-07683ecdf9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.557085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a715b1-9bf8-4368-b9ef-07683ecdf9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.557464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a715b1-9bf8-4368-b9ef-07683ecdf9e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.565586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a715b1-9bf8-4368-b9ef-07683ecdf9e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:15.567199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a715b1-9bf8-4368-b9ef-07683ecdf9e3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-18a715b1-9bf8-4368-b9ef-07683ecdf9e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:15.582725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:15.583658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:15.584581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:15.584581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:15.584581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:15.584581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:15.584581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:15.584581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:15.584581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:15.584581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:15.589280 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 322/639] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:15.590634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bf0cf-2553-47b5-8347-6be17a455768 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.590933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bf0cf-2553-47b5-8347-6be17a455768 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.591220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bf0cf-2553-47b5-8347-6be17a455768 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.594563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 323/640] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:15.595806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bf0cf-2553-47b5-8347-6be17a455768 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:15.651028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bc773-f50f-4c38-bf3c-be264340aee4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:15.659857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:15.686697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a53c4c-2e83-4d19-abf9-1347314099f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:15.687529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a53c4c-2e83-4d19-abf9-1347314099f0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.687823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a53c4c-2e83-4d19-abf9-1347314099f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.688107 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a53c4c-2e83-4d19-abf9-1347314099f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.699551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a53c4c-2e83-4d19-abf9-1347314099f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:15.701020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a53c4c-2e83-4d19-abf9-1347314099f0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:15.704373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bf0cf-2553-47b5-8347-6be17a455768 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:15.706853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e6a53c4c-2e83-4d19-abf9-1347314099f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:15.719222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:15.720364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:15.721473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:15.721473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:15.721473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:15.721473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:15.721473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 318/641] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:15.721473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a53c4c-2e83-4d19-abf9-1347314099f0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:15.742654 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad34c1b-7071-41d5-954f-e248ff7bec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:15.743318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34c1b-7071-41d5-954f-e248ff7bec32 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.743606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34c1b-7071-41d5-954f-e248ff7bec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.743886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad34c1b-7071-41d5-954f-e248ff7bec32 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.744628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad34c1b-7071-41d5-954f-e248ff7bec32 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:15.744940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad34c1b-7071-41d5-954f-e248ff7bec32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:15.746335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad34c1b-7071-41d5-954f-e248ff7bec32 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:15.758950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 324/642] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:15.758950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd59a5-d3e0-42ec-a455-7538e47bc0b4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.758950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd59a5-d3e0-42ec-a455-7538e47bc0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.758950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd59a5-d3e0-42ec-a455-7538e47bc0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.773014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-35fd59a5-d3e0-42ec-a455-7538e47bc0b4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:15.774780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-35fd59a5-d3e0-42ec-a455-7538e47bc0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:15.834008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a53c4c-2e83-4d19-abf9-1347314099f0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:15.852261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 325/643] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:15.853220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd59a5-d3e0-42ec-a455-7538e47bc0b4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:15.881191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff04625-e15f-48ff-9041-24b572e2ae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:15.882294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff04625-e15f-48ff-9041-24b572e2ae5b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:15.882645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff04625-e15f-48ff-9041-24b572e2ae5b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:15.883738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff04625-e15f-48ff-9041-24b572e2ae5b None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:15.884548 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff04625-e15f-48ff-9041-24b572e2ae5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:15.884855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff04625-e15f-48ff-9041-24b572e2ae5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:15.886598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff04625-e15f-48ff-9041-24b572e2ae5b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:15.900147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 319/644] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:15.903406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff04625-e15f-48ff-9041-24b572e2ae5b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:15.934223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:15.960623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:15.968544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fd59a5-d3e0-42ec-a455-7538e47bc0b4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.003010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162ddc4-60cb-42d6-8097-165e5fa231ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.004015 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162ddc4-60cb-42d6-8097-165e5fa231ce None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.004554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162ddc4-60cb-42d6-8097-165e5fa231ce None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.004941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162ddc4-60cb-42d6-8097-165e5fa231ce None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.005746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162ddc4-60cb-42d6-8097-165e5fa231ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:16.006131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162ddc4-60cb-42d6-8097-165e5fa231ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:16.007656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162ddc4-60cb-42d6-8097-165e5fa231ce None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:16.024759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 326/645] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:16.024935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff04625-e15f-48ff-9041-24b572e2ae5b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.025033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162ddc4-60cb-42d6-8097-165e5fa231ce None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:16.060719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee60f3b-45ab-49b9-b5d3-db2ec55c2a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.061518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee60f3b-45ab-49b9-b5d3-db2ec55c2a6c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.061907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee60f3b-45ab-49b9-b5d3-db2ec55c2a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.062201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee60f3b-45ab-49b9-b5d3-db2ec55c2a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.073341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee60f3b-45ab-49b9-b5d3-db2ec55c2a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:16.074817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee60f3b-45ab-49b9-b5d3-db2ec55c2a6c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-9ee60f3b-45ab-49b9-b5d3-db2ec55c2a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:16.092622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:16.093528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:16.094624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:16.094624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:16.094624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:16.094624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:16.094624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 320/646] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:16.094624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee60f3b-45ab-49b9-b5d3-db2ec55c2a6c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:16.136429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162ddc4-60cb-42d6-8097-165e5fa231ce None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.166570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:16.171799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602f5e-16d5-49b0-8172-e4ffc7cbcff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.172598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602f5e-16d5-49b0-8172-e4ffc7cbcff2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.172891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602f5e-16d5-49b0-8172-e4ffc7cbcff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.173169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602f5e-16d5-49b0-8172-e4ffc7cbcff2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.184099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd602f5e-16d5-49b0-8172-e4ffc7cbcff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fd602f5e-16d5-49b0-8172-e4ffc7cbcff2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:16.191828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:16.192772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:16.193635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 327/647] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:16.195289 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602f5e-16d5-49b0-8172-e4ffc7cbcff2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:16.203702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee60f3b-45ab-49b9-b5d3-db2ec55c2a6c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.215380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:16.238344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79390709-d0c4-46b0-9545-e37381829ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.239031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79390709-d0c4-46b0-9545-e37381829ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.239326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79390709-d0c4-46b0-9545-e37381829ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.239605 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79390709-d0c4-46b0-9545-e37381829ea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.240424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79390709-d0c4-46b0-9545-e37381829ea6 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=490d773bf21446789b77c5793a518a5a) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:16.240742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79390709-d0c4-46b0-9545-e37381829ea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:16.242166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79390709-d0c4-46b0-9545-e37381829ea6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:16.258472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 321/648] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:16 2022] GET /identity/v3/role_assignments?user.id=490d773bf21446789b77c5793a518a5a&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:16.258472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79390709-d0c4-46b0-9545-e37381829ea6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:16.321653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602f5e-16d5-49b0-8172-e4ffc7cbcff2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.360840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca984b5-712c-4176-8341-cfad03b84bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.361568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca984b5-712c-4176-8341-cfad03b84bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.361852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca984b5-712c-4176-8341-cfad03b84bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.362131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca984b5-712c-4176-8341-cfad03b84bf7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.371106 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca984b5-712c-4176-8341-cfad03b84bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:16.372574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca984b5-712c-4176-8341-cfad03b84bf7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9ca984b5-712c-4176-8341-cfad03b84bf7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:16.390321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:16.391379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:16.392360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 328/649] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:16.392966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca984b5-712c-4176-8341-cfad03b84bf7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:16.405375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79390709-d0c4-46b0-9545-e37381829ea6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.462766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79edbf-a2eb-442a-81b3-336de16c5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.463877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79edbf-a2eb-442a-81b3-336de16c5aee None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.463877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79edbf-a2eb-442a-81b3-336de16c5aee None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.464131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79edbf-a2eb-442a-81b3-336de16c5aee None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.465362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79edbf-a2eb-442a-81b3-336de16c5aee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:16.465758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79edbf-a2eb-442a-81b3-336de16c5aee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:16.467293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79edbf-a2eb-442a-81b3-336de16c5aee None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:16.469911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:16.481803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 322/650] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:16.483009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79edbf-a2eb-442a-81b3-336de16c5aee None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:16.491126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:16.516931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca984b5-712c-4176-8341-cfad03b84bf7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.559502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8decb8-2595-4666-af0e-3a540dcf8234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.560185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8decb8-2595-4666-af0e-3a540dcf8234 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.560477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8decb8-2595-4666-af0e-3a540dcf8234 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.560759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8decb8-2595-4666-af0e-3a540dcf8234 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.561513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8decb8-2595-4666-af0e-3a540dcf8234 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:16.561821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8decb8-2595-4666-af0e-3a540dcf8234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:16.564059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8decb8-2595-4666-af0e-3a540dcf8234 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:16.574201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 329/651] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:16.578990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8decb8-2595-4666-af0e-3a540dcf8234 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:16.606133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79edbf-a2eb-442a-81b3-336de16c5aee None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.644006 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15204007-a6a0-4ae6-b26c-fe4142139486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.644907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15204007-a6a0-4ae6-b26c-fe4142139486 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.645387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15204007-a6a0-4ae6-b26c-fe4142139486 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.645826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15204007-a6a0-4ae6-b26c-fe4142139486 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.646896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15204007-a6a0-4ae6-b26c-fe4142139486 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:16.647353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15204007-a6a0-4ae6-b26c-fe4142139486 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:16.649004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15204007-a6a0-4ae6-b26c-fe4142139486 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:16.677234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 323/652] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 26 20:51:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:16.677234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15204007-a6a0-4ae6-b26c-fe4142139486 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:16.694888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8decb8-2595-4666-af0e-3a540dcf8234 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.736886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c65a50-e0e5-4699-b0c2-b7a294cca912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.737846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c65a50-e0e5-4699-b0c2-b7a294cca912 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.738545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c65a50-e0e5-4699-b0c2-b7a294cca912 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.738683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c65a50-e0e5-4699-b0c2-b7a294cca912 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.747745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c65a50-e0e5-4699-b0c2-b7a294cca912 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:16.749417 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c65a50-e0e5-4699-b0c2-b7a294cca912 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:16.749837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:16.752045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-47c65a50-e0e5-4699-b0c2-b7a294cca912 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:16.768245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:16.769328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:16.770326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:16.770326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:16.770326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:16.770326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:16.770326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:16.770326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:16.770326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:16.770326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:16.770326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:51:16.770326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:16.773194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 330/653] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:16.774117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c65a50-e0e5-4699-b0c2-b7a294cca912 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:16.803161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15204007-a6a0-4ae6-b26c-fe4142139486 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.840519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6fb44-2102-42a0-8abf-a7def1a63d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.842076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6fb44-2102-42a0-8abf-a7def1a63d1a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.842578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6fb44-2102-42a0-8abf-a7def1a63d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.843034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6fb44-2102-42a0-8abf-a7def1a63d1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.854838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6fb44-2102-42a0-8abf-a7def1a63d1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:16.856721 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6fb44-2102-42a0-8abf-a7def1a63d1a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e4b6fb44-2102-42a0-8abf-a7def1a63d1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:16.877703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:16.878723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:16.879646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:16.879646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:16.879646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:16.879646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:16.882123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 324/654] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:16.884007 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6fb44-2102-42a0-8abf-a7def1a63d1a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:16.898645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c65a50-e0e5-4699-b0c2-b7a294cca912 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:16.944184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7ebce0-3e9c-4037-a937-15bd1befd9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:16.944845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ebce0-3e9c-4037-a937-15bd1befd9cd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:16.945133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ebce0-3e9c-4037-a937-15bd1befd9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:16.945482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7ebce0-3e9c-4037-a937-15bd1befd9cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:16.996730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6fb44-2102-42a0-8abf-a7def1a63d1a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:17.005375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:17.033801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:17.037934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ebce0-3e9c-4037-a937-15bd1befd9cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:17.038244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d53c2c-8fc1-4d4f-a3e2-c88000f661a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:17.039882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d53c2c-8fc1-4d4f-a3e2-c88000f661a7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:17.040102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7ebce0-3e9c-4037-a937-15bd1befd9cd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:17.040587 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d53c2c-8fc1-4d4f-a3e2-c88000f661a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:17.041064 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d53c2c-8fc1-4d4f-a3e2-c88000f661a7 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:17.042463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d53c2c-8fc1-4d4f-a3e2-c88000f661a7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:17.043177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d53c2c-8fc1-4d4f-a3e2-c88000f661a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:17.044674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d53c2c-8fc1-4d4f-a3e2-c88000f661a7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:17.065959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 325/655] 10.222.0.57 () {56 vars in 1137 bytes} [Thu May 26 20:51:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:17.067075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d53c2c-8fc1-4d4f-a3e2-c88000f661a7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:17.155024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 331/656] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:17.155024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece99c9-1921-4adf-940a-b0768d3768fd None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:17.155024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece99c9-1921-4adf-940a-b0768d3768fd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:17.155024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece99c9-1921-4adf-940a-b0768d3768fd None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:17.155024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 332/657] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:17.155447 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece99c9-1921-4adf-940a-b0768d3768fd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:17.182076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d53c2c-8fc1-4d4f-a3e2-c88000f661a7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:17.222957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8f768-b8a8-4423-a1ab-1d684e63eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:17.223622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8f768-b8a8-4423-a1ab-1d684e63eea4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:17.223908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8f768-b8a8-4423-a1ab-1d684e63eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:17.224186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8f768-b8a8-4423-a1ab-1d684e63eea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:17.224900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8f768-b8a8-4423-a1ab-1d684e63eea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:17.225208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8f768-b8a8-4423-a1ab-1d684e63eea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:17.227081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8f768-b8a8-4423-a1ab-1d684e63eea4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:17.239299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 326/658] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:17.240113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8f768-b8a8-4423-a1ab-1d684e63eea4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:17.266606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:17.270741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:17.273791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece99c9-1921-4adf-940a-b0768d3768fd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:17.310200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514afaf-e549-43db-bf0e-09498ef11659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:17.310877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514afaf-e549-43db-bf0e-09498ef11659 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:17.311167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514afaf-e549-43db-bf0e-09498ef11659 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:17.311584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514afaf-e549-43db-bf0e-09498ef11659 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:17.349573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8f768-b8a8-4423-a1ab-1d684e63eea4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:17.383394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a6470-dda7-4a60-970b-eb74da41f084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:17.384185 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6470-dda7-4a60-970b-eb74da41f084 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:17.384407 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514afaf-e549-43db-bf0e-09498ef11659 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:17.384530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6470-dda7-4a60-970b-eb74da41f084 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:17.384744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6470-dda7-4a60-970b-eb74da41f084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:17.386851 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514afaf-e549-43db-bf0e-09498ef11659 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:17.395427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a6470-dda7-4a60-970b-eb74da41f084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:17.396865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249a6470-dda7-4a60-970b-eb74da41f084 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-249a6470-dda7-4a60-970b-eb74da41f084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:17.415021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:17.415923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:17.416863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:17.416863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:17.416863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:17.416863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:17.416863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 327/659] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:17.417425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecb576-b716-4e6e-b658-d6c1f4240f16 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:17.417584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecb576-b716-4e6e-b658-d6c1f4240f16 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:17.417869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecb576-b716-4e6e-b658-d6c1f4240f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:17.434106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a5ecb576-b716-4e6e-b658-d6c1f4240f16 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:17.435322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a5ecb576-b716-4e6e-b658-d6c1f4240f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:17.492366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:17.502627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 333/660] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:17.503447 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514afaf-e549-43db-bf0e-09498ef11659 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:17.519748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 328/661] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:17.520462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecb576-b716-4e6e-b658-d6c1f4240f16 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:17.529545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:17.634573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514afaf-e549-43db-bf0e-09498ef11659 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:17.635026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecb576-b716-4e6e-b658-d6c1f4240f16 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:17.670915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2959f8-4b2d-4046-a0ad-e5af31c8e212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:17.671502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2959f8-4b2d-4046-a0ad-e5af31c8e212 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:17.671787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2959f8-4b2d-4046-a0ad-e5af31c8e212 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:17.672066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2959f8-4b2d-4046-a0ad-e5af31c8e212 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:17.672783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2959f8-4b2d-4046-a0ad-e5af31c8e212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:17.672893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c9eb1-170e-4c5f-8733-486a34503f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:17.673085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2959f8-4b2d-4046-a0ad-e5af31c8e212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:17.673667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c9eb1-170e-4c5f-8733-486a34503f97 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:17.673956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c9eb1-170e-4c5f-8733-486a34503f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:17.674237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c9eb1-170e-4c5f-8733-486a34503f97 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:17.674471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2959f8-4b2d-4046-a0ad-e5af31c8e212 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:17.682085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c9eb1-170e-4c5f-8733-486a34503f97 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:17.683689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c9eb1-170e-4c5f-8733-486a34503f97 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:17.686290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 329/662] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:17.687238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2959f8-4b2d-4046-a0ad-e5af31c8e212 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-787c9eb1-170e-4c5f-8733-486a34503f97 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:17.708152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:17.709175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 334/663] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:17.715807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c9eb1-170e-4c5f-8733-486a34503f97 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:17.759813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:17.774902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:17.797710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2959f8-4b2d-4046-a0ad-e5af31c8e212 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:17.824741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c9eb1-170e-4c5f-8733-486a34503f97 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:17.835576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79611322-d638-4a75-bea3-0c6344b4d25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:17.836902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79611322-d638-4a75-bea3-0c6344b4d25a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:17.837188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79611322-d638-4a75-bea3-0c6344b4d25a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:17.837470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79611322-d638-4a75-bea3-0c6344b4d25a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:17.848667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79611322-d638-4a75-bea3-0c6344b4d25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:17.850103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79611322-d638-4a75-bea3-0c6344b4d25a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:17.860452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bed41-b36c-4bfc-bcfc-ee0c84d79547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:17.861394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bed41-b36c-4bfc-bcfc-ee0c84d79547 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:17.861841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bed41-b36c-4bfc-bcfc-ee0c84d79547 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:17.862258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bed41-b36c-4bfc-bcfc-ee0c84d79547 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-79611322-d638-4a75-bea3-0c6344b4d25a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:17.867225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:17.868249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:17.869385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:17.869385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:17.869385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:17.869385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:17.872935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 330/664] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:17.873760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79611322-d638-4a75-bea3-0c6344b4d25a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:17.875972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bed41-b36c-4bfc-bcfc-ee0c84d79547 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-285bed41-b36c-4bfc-bcfc-ee0c84d79547 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:17.884532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:17.885645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:17.886774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:17.893480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 335/665] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:17.893480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bed41-b36c-4bfc-bcfc-ee0c84d79547 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:17.988788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79611322-d638-4a75-bea3-0c6344b4d25a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:17.997430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bed41-b36c-4bfc-bcfc-ee0c84d79547 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:18.007013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:18.017048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:18.026907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4b4f-df9c-472a-a31f-f5b393d32cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.027713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4b4f-df9c-472a-a31f-f5b393d32cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.027990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4b4f-df9c-472a-a31f-f5b393d32cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.028268 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f4b4f-df9c-472a-a31f-f5b393d32cf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.035096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f4b4f-df9c-472a-a31f-f5b393d32cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.035674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f4f3d-8a88-4c88-a2a2-ad9b0d87be7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.036339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f4f3d-8a88-4c88-a2a2-ad9b0d87be7e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.036538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f4b4f-df9c-472a-a31f-f5b393d32cf8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.036640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f4f3d-8a88-4c88-a2a2-ad9b0d87be7e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.036893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f4f3d-8a88-4c88-a2a2-ad9b0d87be7e None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.037710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f4f3d-8a88-4c88-a2a2-ad9b0d87be7e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:18.038021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f4f3d-8a88-4c88-a2a2-ad9b0d87be7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.039580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f4f3d-8a88-4c88-a2a2-ad9b0d87be7e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-792f4b4f-df9c-472a-a31f-f5b393d32cf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:18.050861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:18.051822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:18.052740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:18.052740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:18.052740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:18.052740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:18.052740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:18.052740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:18.052740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:18.052740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:18.055277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 331/666] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:18.056623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4b4f-df9c-472a-a31f-f5b393d32cf8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.057915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 336/667] 10.222.0.57 () {56 vars in 1139 bytes} [Thu May 26 20:51:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:18.058768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f4f3d-8a88-4c88-a2a2-ad9b0d87be7e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.161168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f4f3d-8a88-4c88-a2a2-ad9b0d87be7e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:18.161545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f4b4f-df9c-472a-a31f-f5b393d32cf8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:18.198548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fe590-2f8b-43bf-a801-c800e1bf330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.199219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fe590-2f8b-43bf-a801-c800e1bf330a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.199508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fe590-2f8b-43bf-a801-c800e1bf330a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.199609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3ac94-2331-4baa-b0d6-892dddba5785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.199799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fe590-2f8b-43bf-a801-c800e1bf330a None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.200210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ac94-2331-4baa-b0d6-892dddba5785 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.200492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ac94-2331-4baa-b0d6-892dddba5785 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.200586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fe590-2f8b-43bf-a801-c800e1bf330a None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:18.200767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ac94-2331-4baa-b0d6-892dddba5785 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.200861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fe590-2f8b-43bf-a801-c800e1bf330a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.201486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3ac94-2331-4baa-b0d6-892dddba5785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:18.201790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3ac94-2331-4baa-b0d6-892dddba5785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.202191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fe590-2f8b-43bf-a801-c800e1bf330a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.203172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3ac94-2331-4baa-b0d6-892dddba5785 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.215404 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 337/668] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:18.215404 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fe590-2f8b-43bf-a801-c800e1bf330a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.224521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 332/669] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:18.225294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3ac94-2331-4baa-b0d6-892dddba5785 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.251323 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:18.269585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:18.316045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fe590-2f8b-43bf-a801-c800e1bf330a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:18.335153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3ac94-2331-4baa-b0d6-892dddba5785 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:18.349290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37d47b-4cff-4e51-a170-1d144267d5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.349951 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37d47b-4cff-4e51-a170-1d144267d5af None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.350240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37d47b-4cff-4e51-a170-1d144267d5af None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.350554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37d47b-4cff-4e51-a170-1d144267d5af None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.351356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37d47b-4cff-4e51-a170-1d144267d5af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:18.351702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37d47b-4cff-4e51-a170-1d144267d5af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.353171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37d47b-4cff-4e51-a170-1d144267d5af None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.364979 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 338/670] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:18.364979 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37d47b-4cff-4e51-a170-1d144267d5af None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.376715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78364c0f-220b-489e-b596-b9bc1fde5690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.377397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78364c0f-220b-489e-b596-b9bc1fde5690 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.377665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78364c0f-220b-489e-b596-b9bc1fde5690 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.377937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78364c0f-220b-489e-b596-b9bc1fde5690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.454360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78364c0f-220b-489e-b596-b9bc1fde5690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.455889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78364c0f-220b-489e-b596-b9bc1fde5690 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.470559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:18.474478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37d47b-4cff-4e51-a170-1d144267d5af None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:18.499855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:18.518103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d78991-48a6-4f60-a6bd-35530dfc2da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.518854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d78991-48a6-4f60-a6bd-35530dfc2da5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.519142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d78991-48a6-4f60-a6bd-35530dfc2da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.519599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d78991-48a6-4f60-a6bd-35530dfc2da5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.529821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d78991-48a6-4f60-a6bd-35530dfc2da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.531459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d78991-48a6-4f60-a6bd-35530dfc2da5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b7d78991-48a6-4f60-a6bd-35530dfc2da5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:18.549670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:18.550630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:18.551534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:18.551534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:18.551534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:18.551534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:18.551534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 339/671] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:18.552414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d78991-48a6-4f60-a6bd-35530dfc2da5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.571871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 333/672] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:18.571871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78364c0f-220b-489e-b596-b9bc1fde5690 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.665313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d78991-48a6-4f60-a6bd-35530dfc2da5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:18.666796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78364c0f-220b-489e-b596-b9bc1fde5690 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:18.698692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984a08b-00bb-482d-9ce0-99ed7d0c0338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.699481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984a08b-00bb-482d-9ce0-99ed7d0c0338 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.699760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984a08b-00bb-482d-9ce0-99ed7d0c0338 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.700035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984a08b-00bb-482d-9ce0-99ed7d0c0338 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.700139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509eefc7-0f83-4cf6-a840-270d25e519bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.700731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eefc7-0f83-4cf6-a840-270d25e519bc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.701020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eefc7-0f83-4cf6-a840-270d25e519bc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.701300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eefc7-0f83-4cf6-a840-270d25e519bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.702126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509eefc7-0f83-4cf6-a840-270d25e519bc None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=2c3bd8dd462846bf9423f8528aa5c489) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:18.702464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509eefc7-0f83-4cf6-a840-270d25e519bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.703917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509eefc7-0f83-4cf6-a840-270d25e519bc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.706694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984a08b-00bb-482d-9ce0-99ed7d0c0338 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.708119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984a08b-00bb-482d-9ce0-99ed7d0c0338 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.709707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 334/673] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/role_assignments?user.id=2c3bd8dd462846bf9423f8528aa5c489&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:18.710507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509eefc7-0f83-4cf6-a840-270d25e519bc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.716561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0984a08b-00bb-482d-9ce0-99ed7d0c0338 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:18.722212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:18.723257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:18.724164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:18.724164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:18.724164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:18.724164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:18.724164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:18.724164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:18.724164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:18.724164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:18.724164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 340/674] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:18.725785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984a08b-00bb-482d-9ce0-99ed7d0c0338 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.745844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:18.834758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509eefc7-0f83-4cf6-a840-270d25e519bc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:18.865025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984a08b-00bb-482d-9ce0-99ed7d0c0338 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:18.873030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46154524-5e59-4f9d-b6b8-97c6a3bedb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.873600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46154524-5e59-4f9d-b6b8-97c6a3bedb09 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.873693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46154524-5e59-4f9d-b6b8-97c6a3bedb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.873922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46154524-5e59-4f9d-b6b8-97c6a3bedb09 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.874672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46154524-5e59-4f9d-b6b8-97c6a3bedb09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:18.874984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46154524-5e59-4f9d-b6b8-97c6a3bedb09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.876371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46154524-5e59-4f9d-b6b8-97c6a3bedb09 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.890106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 335/675] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:18.891349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff197-6cb9-4e2b-a628-ae7e688282d0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.891576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff197-6cb9-4e2b-a628-ae7e688282d0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.891855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ff197-6cb9-4e2b-a628-ae7e688282d0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.893174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 336/676] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:18.893772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff197-6cb9-4e2b-a628-ae7e688282d0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.905898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb60662-18c5-486e-b631-60b7fa0979a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:18.907680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb60662-18c5-486e-b631-60b7fa0979a6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:18.907963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb60662-18c5-486e-b631-60b7fa0979a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:18.908237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb60662-18c5-486e-b631-60b7fa0979a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:18.920169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb60662-18c5-486e-b631-60b7fa0979a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:18.921614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb60662-18c5-486e-b631-60b7fa0979a6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-bbb60662-18c5-486e-b631-60b7fa0979a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:18.939910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:18.941116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:18.942212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:18.942212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:18.942212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:18.942212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:18.942212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 341/677] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:18.942212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb60662-18c5-486e-b631-60b7fa0979a6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:18.965176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:19.009741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ff197-6cb9-4e2b-a628-ae7e688282d0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:19.017567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:19.046629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83882a19-083f-41ca-84e0-cd581324cb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:19.047290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83882a19-083f-41ca-84e0-cd581324cb65 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:19.047575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83882a19-083f-41ca-84e0-cd581324cb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:19.047854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83882a19-083f-41ca-84e0-cd581324cb65 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:19.048630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83882a19-083f-41ca-84e0-cd581324cb65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:19.048939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83882a19-083f-41ca-84e0-cd581324cb65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:19.050358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83882a19-083f-41ca-84e0-cd581324cb65 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:19.060202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb60662-18c5-486e-b631-60b7fa0979a6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:19.062986 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 337/678] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:19.064151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a270e-7677-4dcd-89d9-bf06da87ad07 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:19.064493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a270e-7677-4dcd-89d9-bf06da87ad07 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:19.064774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a270e-7677-4dcd-89d9-bf06da87ad07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:19.082414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e14a270e-7677-4dcd-89d9-bf06da87ad07 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:19.083619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e14a270e-7677-4dcd-89d9-bf06da87ad07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:19.100742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1281c08-f019-46ad-8b87-c7508df46e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:19.102093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281c08-f019-46ad-8b87-c7508df46e70 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:19.102385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281c08-f019-46ad-8b87-c7508df46e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:19.102771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1281c08-f019-46ad-8b87-c7508df46e70 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:19.111023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1281c08-f019-46ad-8b87-c7508df46e70 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:19.112601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1281c08-f019-46ad-8b87-c7508df46e70 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f1281c08-f019-46ad-8b87-c7508df46e70 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:19.126716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:19.127738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:19.128695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 342/679] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:19.129312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1281c08-f019-46ad-8b87-c7508df46e70 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:19.162520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 338/680] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:19.162520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a270e-7677-4dcd-89d9-bf06da87ad07 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:19.225763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:19.239669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1281c08-f019-46ad-8b87-c7508df46e70 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:19.262978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:19.276239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134cca9-3d76-4111-ad7f-d334bf9f3892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:19.276898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134cca9-3d76-4111-ad7f-d334bf9f3892 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:19.277184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134cca9-3d76-4111-ad7f-d334bf9f3892 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:19.277460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7134cca9-3d76-4111-ad7f-d334bf9f3892 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:19.278182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134cca9-3d76-4111-ad7f-d334bf9f3892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:19.278527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134cca9-3d76-4111-ad7f-d334bf9f3892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:19.279953 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7134cca9-3d76-4111-ad7f-d334bf9f3892 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:19.281382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a270e-7677-4dcd-89d9-bf06da87ad07 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:19.293023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 343/681] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:19.294567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134cca9-3d76-4111-ad7f-d334bf9f3892 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:19.326280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f49fc-0252-485f-a031-be743994db14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:19.327329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f49fc-0252-485f-a031-be743994db14 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:19.327807 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f49fc-0252-485f-a031-be743994db14 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:19.328335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f49fc-0252-485f-a031-be743994db14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:19.329164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f49fc-0252-485f-a031-be743994db14 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=afe351334485400a8c11f17492c9ff5a) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:19.329477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f49fc-0252-485f-a031-be743994db14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:19.331027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f49fc-0252-485f-a031-be743994db14 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:19.338089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 339/682] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:19 2022] GET /identity/v3/role_assignments?user.id=afe351334485400a8c11f17492c9ff5a&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:19.338896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f49fc-0252-485f-a031-be743994db14 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:19.414720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7134cca9-3d76-4111-ad7f-d334bf9f3892 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:19.454609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dbbee-8117-4bd9-ba2a-9a79eab02a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:19.455693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dbbee-8117-4bd9-ba2a-9a79eab02a88 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:19.456214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dbbee-8117-4bd9-ba2a-9a79eab02a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:19.456711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dbbee-8117-4bd9-ba2a-9a79eab02a88 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:19.457766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dbbee-8117-4bd9-ba2a-9a79eab02a88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:19.458286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dbbee-8117-4bd9-ba2a-9a79eab02a88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:19.459991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497dbbee-8117-4bd9-ba2a-9a79eab02a88 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:19.482048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 344/683] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 26 20:51:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:19.483371 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dbbee-8117-4bd9-ba2a-9a79eab02a88 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:19.493134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f49fc-0252-485f-a031-be743994db14 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:19.522280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:19.541732 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:19.543602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9127f7b-8469-4e62-819a-124ec3949e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:19.544414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9127f7b-8469-4e62-819a-124ec3949e24 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:19.544696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9127f7b-8469-4e62-819a-124ec3949e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:19.544975 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9127f7b-8469-4e62-819a-124ec3949e24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:19.559000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9127f7b-8469-4e62-819a-124ec3949e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f9127f7b-8469-4e62-819a-124ec3949e24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:19.567002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:19.568169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:19.569211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:19.573135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 340/684] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:19.574219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9127f7b-8469-4e62-819a-124ec3949e24 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:19.621620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dbbee-8117-4bd9-ba2a-9a79eab02a88 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:19.658555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e60e8-69cf-4fa1-9927-aa8d8c01fe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:19.659349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e60e8-69cf-4fa1-9927-aa8d8c01fe4e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:19.659634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e60e8-69cf-4fa1-9927-aa8d8c01fe4e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:19.660004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e60e8-69cf-4fa1-9927-aa8d8c01fe4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:19.667309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e60e8-69cf-4fa1-9927-aa8d8c01fe4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:19.668814 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e60e8-69cf-4fa1-9927-aa8d8c01fe4e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6e2e60e8-69cf-4fa1-9927-aa8d8c01fe4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:19.683783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:19.684672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:19.685971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:19.685971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:19.685971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:19.685971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:19.685971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:19.685971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:19.685971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:19.685971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:19.685971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 345/685] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:19.686477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e60e8-69cf-4fa1-9927-aa8d8c01fe4e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:19.690059 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9127f7b-8469-4e62-819a-124ec3949e24 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:19.725986 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66b15b-da67-4322-a963-e9dc612067d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:19.726695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66b15b-da67-4322-a963-e9dc612067d2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:19.726983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66b15b-da67-4322-a963-e9dc612067d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:19.727259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66b15b-da67-4322-a963-e9dc612067d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:19.776945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:19.807822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66b15b-da67-4322-a963-e9dc612067d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:19.810583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66b15b-da67-4322-a963-e9dc612067d2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:19.821437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:19.824458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e60e8-69cf-4fa1-9927-aa8d8c01fe4e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:19.864483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dda24-4152-43da-800d-ce7485b449ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:19.865935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dda24-4152-43da-800d-ce7485b449ba None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:19.866847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dda24-4152-43da-800d-ce7485b449ba None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:19.867717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099dda24-4152-43da-800d-ce7485b449ba None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:19.869055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dda24-4152-43da-800d-ce7485b449ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:19.869530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dda24-4152-43da-800d-ce7485b449ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:19.871392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099dda24-4152-43da-800d-ce7485b449ba None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:19.888708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 346/686] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:19.888708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dda24-4152-43da-800d-ce7485b449ba None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:19.949807 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 341/687] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:19.950702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66b15b-da67-4322-a963-e9dc612067d2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.013760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099dda24-4152-43da-800d-ce7485b449ba None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.035263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:20.049687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74973d8c-1ae8-4c17-8270-a99881db9ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.050678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74973d8c-1ae8-4c17-8270-a99881db9ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.051168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74973d8c-1ae8-4c17-8270-a99881db9ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.051675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74973d8c-1ae8-4c17-8270-a99881db9ed1 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.052652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74973d8c-1ae8-4c17-8270-a99881db9ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:20.053163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74973d8c-1ae8-4c17-8270-a99881db9ed1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:20.054885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74973d8c-1ae8-4c17-8270-a99881db9ed1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:20.059975 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66b15b-da67-4322-a963-e9dc612067d2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.068086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 347/688] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:20.069134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74973d8c-1ae8-4c17-8270-a99881db9ed1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.094554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:20.096580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773a201-c5aa-42f9-be05-711ce0a4075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.097196 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773a201-c5aa-42f9-be05-711ce0a4075b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.097821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773a201-c5aa-42f9-be05-711ce0a4075b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.097821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773a201-c5aa-42f9-be05-711ce0a4075b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.108834 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773a201-c5aa-42f9-be05-711ce0a4075b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:20.110295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773a201-c5aa-42f9-be05-711ce0a4075b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3773a201-c5aa-42f9-be05-711ce0a4075b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:20.128431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:20.129345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:20.130593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:20.130593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:20.130593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:20.130593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:20.130593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 342/689] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:20.131033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773a201-c5aa-42f9-be05-711ce0a4075b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.179773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74973d8c-1ae8-4c17-8270-a99881db9ed1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.215672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c28e4-3f65-4f54-9f8d-2ce36cde4799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.216744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c28e4-3f65-4f54-9f8d-2ce36cde4799 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.217099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c28e4-3f65-4f54-9f8d-2ce36cde4799 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.217450 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c28e4-3f65-4f54-9f8d-2ce36cde4799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.228618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c28e4-3f65-4f54-9f8d-2ce36cde4799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:20.230263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c28e4-3f65-4f54-9f8d-2ce36cde4799 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:20.234970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773a201-c5aa-42f9-be05-711ce0a4075b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-556c28e4-3f65-4f54-9f8d-2ce36cde4799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:20.249093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:20.250905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:20.253162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:20.253162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:20.253162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:20.253162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:20.253162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 348/690] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:20.253162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c28e4-3f65-4f54-9f8d-2ce36cde4799 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.262687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:20.271234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19fe00-fd45-4509-bddc-0dd0ef013d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.271967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19fe00-fd45-4509-bddc-0dd0ef013d09 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.272580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19fe00-fd45-4509-bddc-0dd0ef013d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.273160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b19fe00-fd45-4509-bddc-0dd0ef013d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.274971 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19fe00-fd45-4509-bddc-0dd0ef013d09 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=94b8d8b089264b5192a651b41bf2a205) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:20.275441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19fe00-fd45-4509-bddc-0dd0ef013d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:20.277036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b19fe00-fd45-4509-bddc-0dd0ef013d09 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:20.283771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 343/691] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:20 2022] GET /identity/v3/role_assignments?user.id=94b8d8b089264b5192a651b41bf2a205&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:20.284759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19fe00-fd45-4509-bddc-0dd0ef013d09 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.338026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:20.367597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c28e4-3f65-4f54-9f8d-2ce36cde4799 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.399897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b19fe00-fd45-4509-bddc-0dd0ef013d09 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.405915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16fb72-a303-46ac-ba7b-59a7ec5516aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.406629 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16fb72-a303-46ac-ba7b-59a7ec5516aa None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.406917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16fb72-a303-46ac-ba7b-59a7ec5516aa None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.407194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16fb72-a303-46ac-ba7b-59a7ec5516aa None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.408068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16fb72-a303-46ac-ba7b-59a7ec5516aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:20.408382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16fb72-a303-46ac-ba7b-59a7ec5516aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:20.410013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce16fb72-a303-46ac-ba7b-59a7ec5516aa None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:20.423554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 349/692] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:20.429062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16fb72-a303-46ac-ba7b-59a7ec5516aa None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.437294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90717db7-aec8-45f6-863e-5bb0b9ffd684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.437903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717db7-aec8-45f6-863e-5bb0b9ffd684 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.438194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717db7-aec8-45f6-863e-5bb0b9ffd684 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.438506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90717db7-aec8-45f6-863e-5bb0b9ffd684 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.439378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717db7-aec8-45f6-863e-5bb0b9ffd684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:20.439699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717db7-aec8-45f6-863e-5bb0b9ffd684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:20.441127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90717db7-aec8-45f6-863e-5bb0b9ffd684 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:20.455827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 344/693] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:20.461360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90717db7-aec8-45f6-863e-5bb0b9ffd684 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.521020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:20.541082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16fb72-a303-46ac-ba7b-59a7ec5516aa None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.575600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90717db7-aec8-45f6-863e-5bb0b9ffd684 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.579605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589599fd-5787-4096-af88-e5441b6dff58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.580480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589599fd-5787-4096-af88-e5441b6dff58 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.580761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589599fd-5787-4096-af88-e5441b6dff58 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.581090 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589599fd-5787-4096-af88-e5441b6dff58 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.583352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:20.590625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589599fd-5787-4096-af88-e5441b6dff58 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:20.592246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589599fd-5787-4096-af88-e5441b6dff58 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-589599fd-5787-4096-af88-e5441b6dff58 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:20.609145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:20.611944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:20.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:20.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:20.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:20.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:20.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:20.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:20.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:20.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:20.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:20.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:20.615575 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 350/694] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 26 20:51:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:20.618479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589599fd-5787-4096-af88-e5441b6dff58 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.656159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69043f57-8758-4eae-ab07-41c7efb6badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.656961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69043f57-8758-4eae-ab07-41c7efb6badc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.658209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69043f57-8758-4eae-ab07-41c7efb6badc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.658209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69043f57-8758-4eae-ab07-41c7efb6badc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.670098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69043f57-8758-4eae-ab07-41c7efb6badc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:20.671859 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69043f57-8758-4eae-ab07-41c7efb6badc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-69043f57-8758-4eae-ab07-41c7efb6badc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:20.691524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:20.692500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:20.693414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:20.693414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:20.693414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:20.693414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:20.693414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:20.693414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:20.693414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:20.693414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:20.696806 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 345/695] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:20.702301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69043f57-8758-4eae-ab07-41c7efb6badc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.754949 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589599fd-5787-4096-af88-e5441b6dff58 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.792500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc395-5abf-4a5b-9272-039b6cc3a65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.793536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc395-5abf-4a5b-9272-039b6cc3a65a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.794035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc395-5abf-4a5b-9272-039b6cc3a65a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.794585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dc395-5abf-4a5b-9272-039b6cc3a65a None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.795622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dc395-5abf-4a5b-9272-039b6cc3a65a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:20.796206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dc395-5abf-4a5b-9272-039b6cc3a65a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:20.797856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dc395-5abf-4a5b-9272-039b6cc3a65a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:20.819167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69043f57-8758-4eae-ab07-41c7efb6badc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.822049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 351/696] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 26 20:51:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:20.827770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc395-5abf-4a5b-9272-039b6cc3a65a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.832879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:20.855676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82227cc-09b8-4469-9efb-bfffbe017595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.856348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82227cc-09b8-4469-9efb-bfffbe017595 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.856631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82227cc-09b8-4469-9efb-bfffbe017595 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.856916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82227cc-09b8-4469-9efb-bfffbe017595 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.857707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82227cc-09b8-4469-9efb-bfffbe017595 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:20.858024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82227cc-09b8-4469-9efb-bfffbe017595 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:20.860440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82227cc-09b8-4469-9efb-bfffbe017595 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:20.873896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 346/697] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:20.879570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82227cc-09b8-4469-9efb-bfffbe017595 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:20.891432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:20.949997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dc395-5abf-4a5b-9272-039b6cc3a65a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:20.987940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b22bf3-fc24-49d8-9ee1-aac5452b4ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:20.988889 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b22bf3-fc24-49d8-9ee1-aac5452b4ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:20.989354 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b22bf3-fc24-49d8-9ee1-aac5452b4ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:20.989832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b22bf3-fc24-49d8-9ee1-aac5452b4ab4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:20.990115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82227cc-09b8-4469-9efb-bfffbe017595 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:21.026574 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20b78a-24d9-4974-be65-d6c86e2c8160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:21.027417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20b78a-24d9-4974-be65-d6c86e2c8160 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:21.027798 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20b78a-24d9-4974-be65-d6c86e2c8160 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:21.028156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20b78a-24d9-4974-be65-d6c86e2c8160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:21.029094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20b78a-24d9-4974-be65-d6c86e2c8160 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=1285406084604bd19287de9e1beeeaa8) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:21.029490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20b78a-24d9-4974-be65-d6c86e2c8160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:21.031167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20b78a-24d9-4974-be65-d6c86e2c8160 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:21.038588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 347/698] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:20 2022] GET /identity/v3/role_assignments?user.id=1285406084604bd19287de9e1beeeaa8&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:21.072584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b22bf3-fc24-49d8-9ee1-aac5452b4ab4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:21.074388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b22bf3-fc24-49d8-9ee1-aac5452b4ab4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:21.131817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:21.195659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 352/699] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:21.201661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20b78a-24d9-4974-be65-d6c86e2c8160 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:21.268315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:21.278691 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752ff5b-f54b-46ee-a499-c46e42e638d5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:21.279020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752ff5b-f54b-46ee-a499-c46e42e638d5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:21.279436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752ff5b-f54b-46ee-a499-c46e42e638d5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:21.281244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 353/700] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:21.287380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28a483-07a9-49fe-855e-2e1cf6ddb2fb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:21.287694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28a483-07a9-49fe-855e-2e1cf6ddb2fb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:21.287950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28a483-07a9-49fe-855e-2e1cf6ddb2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:21.307978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-bf28a483-07a9-49fe-855e-2e1cf6ddb2fb None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:21.309677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bf28a483-07a9-49fe-855e-2e1cf6ddb2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:21.362337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20b78a-24d9-4974-be65-d6c86e2c8160 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:21.391091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 354/701] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:51:21.405729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c91ffa-d317-4f2b-8be9-274e075386d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:21.406567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c91ffa-d317-4f2b-8be9-274e075386d9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:21.406906 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c91ffa-d317-4f2b-8be9-274e075386d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:21.407186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c91ffa-d317-4f2b-8be9-274e075386d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:21.418672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c91ffa-d317-4f2b-8be9-274e075386d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:21.420202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c91ffa-d317-4f2b-8be9-274e075386d9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:21.420939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28a483-07a9-49fe-855e-2e1cf6ddb2fb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e5c91ffa-d317-4f2b-8be9-274e075386d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:21.442725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:21.443749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:21.444649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:21.444649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:21.444649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:21.444649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:21.444649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 348/702] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:21.450341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c91ffa-d317-4f2b-8be9-274e075386d9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:21.520519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:21.545542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28a483-07a9-49fe-855e-2e1cf6ddb2fb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:21.572751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:21.587735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747f756-97fa-49be-b2fe-27258f6863b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:21.588856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747f756-97fa-49be-b2fe-27258f6863b3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:21.589373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747f756-97fa-49be-b2fe-27258f6863b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:21.589851 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747f756-97fa-49be-b2fe-27258f6863b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:21.589955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c91ffa-d317-4f2b-8be9-274e075386d9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:21.601859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747f756-97fa-49be-b2fe-27258f6863b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:21.603885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747f756-97fa-49be-b2fe-27258f6863b3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1747f756-97fa-49be-b2fe-27258f6863b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:21.624772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:21.625751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:21.626712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:21.626712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:21.626712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:21.626712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:21.629101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 355/703] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:21.635980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747f756-97fa-49be-b2fe-27258f6863b3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:21.638153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4000b-5ab7-41dc-8f36-ab9a14685d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:21.638858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4000b-5ab7-41dc-8f36-ab9a14685d87 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:21.639145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4000b-5ab7-41dc-8f36-ab9a14685d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:21.639429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4000b-5ab7-41dc-8f36-ab9a14685d87 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:21.640398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4000b-5ab7-41dc-8f36-ab9a14685d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:21.640713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4000b-5ab7-41dc-8f36-ab9a14685d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:21.643037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4000b-5ab7-41dc-8f36-ab9a14685d87 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:21.663688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 349/704] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:21.663925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4000b-5ab7-41dc-8f36-ab9a14685d87 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:21.769904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747f756-97fa-49be-b2fe-27258f6863b3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:21.797505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4000b-5ab7-41dc-8f36-ab9a14685d87 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:21.821163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d124c-e56a-49aa-9df3-bb86b7323e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:21.821163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d124c-e56a-49aa-9df3-bb86b7323e9c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:21.821163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d124c-e56a-49aa-9df3-bb86b7323e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:21.821163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d124c-e56a-49aa-9df3-bb86b7323e9c None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:21.821564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d124c-e56a-49aa-9df3-bb86b7323e9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:21.821564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d124c-e56a-49aa-9df3-bb86b7323e9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:21.821564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d124c-e56a-49aa-9df3-bb86b7323e9c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:21.822577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:21.833221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 356/705] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:21.843969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d124c-e56a-49aa-9df3-bb86b7323e9c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:21.860971 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a1658-1805-41a8-a55e-0cbebedcd5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:21.862204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a1658-1805-41a8-a55e-0cbebedcd5ab None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:21.862719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a1658-1805-41a8-a55e-0cbebedcd5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:21.863234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a1658-1805-41a8-a55e-0cbebedcd5ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:21.879714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a1658-1805-41a8-a55e-0cbebedcd5ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:21.881799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a1658-1805-41a8-a55e-0cbebedcd5ab None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-253a1658-1805-41a8-a55e-0cbebedcd5ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:21.911759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:21.912690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:21.913584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:21.913584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:21.913584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:21.913584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:21.913584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:21.913584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:21.913584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:21.913584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:21.916221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 350/706] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:21.924118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a1658-1805-41a8-a55e-0cbebedcd5ab None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:21.939848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:21.960171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d124c-e56a-49aa-9df3-bb86b7323e9c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:22.006930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a601556f-b1f6-4290-9093-b7e5da706ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:22.007667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601556f-b1f6-4290-9093-b7e5da706ff7 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:22.008451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601556f-b1f6-4290-9093-b7e5da706ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:22.008729 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601556f-b1f6-4290-9093-b7e5da706ff7 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:22.009448 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601556f-b1f6-4290-9093-b7e5da706ff7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:22.011205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a601556f-b1f6-4290-9093-b7e5da706ff7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:22.035135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 357/707] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:51:21 2022] POST /identity/v3/users => generated 254 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:51:22.036634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b648878-990f-4a8f-b2bf-9f0284fcb1fa None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:22.036985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b648878-990f-4a8f-b2bf-9f0284fcb1fa None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:22.037294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b648878-990f-4a8f-b2bf-9f0284fcb1fa None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:22.043603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 358/708] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:22.046863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b09ec-dc15-4700-9f19-3a28fa59d5cb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:22.047229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b09ec-dc15-4700-9f19-3a28fa59d5cb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:22.047674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b09ec-dc15-4700-9f19-3a28fa59d5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:22.059940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a1658-1805-41a8-a55e-0cbebedcd5ab None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:22.067170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c60b09ec-dc15-4700-9f19-3a28fa59d5cb None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:22.068510 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c60b09ec-dc15-4700-9f19-3a28fa59d5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:22.102726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceba95-83fe-48f0-907d-ac12e264e531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:22.103634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceba95-83fe-48f0-907d-ac12e264e531 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:22.104060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceba95-83fe-48f0-907d-ac12e264e531 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:22.104477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ceba95-83fe-48f0-907d-ac12e264e531 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:22.105399 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceba95-83fe-48f0-907d-ac12e264e531 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:22.105882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceba95-83fe-48f0-907d-ac12e264e531 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:22.107558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ceba95-83fe-48f0-907d-ac12e264e531 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:22.124733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:22.125154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 351/709] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:22.126382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceba95-83fe-48f0-907d-ac12e264e531 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:22.150283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 359/710] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:22.207498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b09ec-dc15-4700-9f19-3a28fa59d5cb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:22.226940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:22.260122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ceba95-83fe-48f0-907d-ac12e264e531 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:22.296657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed13efc-3da3-4b18-9d2a-bea356235bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:22.297319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed13efc-3da3-4b18-9d2a-bea356235bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:22.297603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed13efc-3da3-4b18-9d2a-bea356235bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:22.297931 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed13efc-3da3-4b18-9d2a-bea356235bc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:22.298742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed13efc-3da3-4b18-9d2a-bea356235bc3 None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=490d773bf21446789b77c5793a518a5a) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:22.299059 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed13efc-3da3-4b18-9d2a-bea356235bc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:22.300485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed13efc-3da3-4b18-9d2a-bea356235bc3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:22.310580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 352/711] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:22 2022] GET /identity/v3/role_assignments?user.id=490d773bf21446789b77c5793a518a5a&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:22.367735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b09ec-dc15-4700-9f19-3a28fa59d5cb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:22.408157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6645c9-db37-4953-8b24-e90708ebea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:22.409385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6645c9-db37-4953-8b24-e90708ebea00 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:22.409921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6645c9-db37-4953-8b24-e90708ebea00 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:22.410397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6645c9-db37-4953-8b24-e90708ebea00 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:22.411312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6645c9-db37-4953-8b24-e90708ebea00 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:22.413094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6645c9-db37-4953-8b24-e90708ebea00 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-db6645c9-db37-4953-8b24-e90708ebea00 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:22.428553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:22.429664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:51:22.430540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:22.433323 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 360/712] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 26 20:51:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:22.439641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed13efc-3da3-4b18-9d2a-bea356235bc3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:22.562316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed13efc-3da3-4b18-9d2a-bea356235bc3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:22.595736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:22.602534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0fd12f-b171-48f3-a2c3-f6444859c93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:22.604087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fd12f-b171-48f3-a2c3-f6444859c93f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:22.604486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fd12f-b171-48f3-a2c3-f6444859c93f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:22.604830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fd12f-b171-48f3-a2c3-f6444859c93f None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:22.605616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fd12f-b171-48f3-a2c3-f6444859c93f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:22.606002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fd12f-b171-48f3-a2c3-f6444859c93f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:22.607487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fd12f-b171-48f3-a2c3-f6444859c93f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:22.614986 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 353/713] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 26 20:51:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:22.621082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6645c9-db37-4953-8b24-e90708ebea00 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:22.678511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:22.743275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6645c9-db37-4953-8b24-e90708ebea00 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:22.790528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a5646-4d74-42d7-b1a7-36e22c2b7170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:22.791285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a5646-4d74-42d7-b1a7-36e22c2b7170 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:22.791639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a5646-4d74-42d7-b1a7-36e22c2b7170 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:22.791989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a5646-4d74-42d7-b1a7-36e22c2b7170 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:22.792765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a5646-4d74-42d7-b1a7-36e22c2b7170 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:22.793142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a5646-4d74-42d7-b1a7-36e22c2b7170 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:22.794672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a5646-4d74-42d7-b1a7-36e22c2b7170 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:22.800832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 361/714] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 26 20:51:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:23.362061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95534aec-8a0e-4f6b-9da5-8443936f9f9b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:23.362656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95534aec-8a0e-4f6b-9da5-8443936f9f9b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:23.362656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95534aec-8a0e-4f6b-9da5-8443936f9f9b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:23.364264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 354/715] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:23.369412 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74987df8-bdaa-4884-bed6-e4c0293f0922 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:23.369879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74987df8-bdaa-4884-bed6-e4c0293f0922 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:23.369965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74987df8-bdaa-4884-bed6-e4c0293f0922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:23.390964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-74987df8-bdaa-4884-bed6-e4c0293f0922 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:23.392154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-74987df8-bdaa-4884-bed6-e4c0293f0922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:23.488260 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 362/716] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:23.522388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95534aec-8a0e-4f6b-9da5-8443936f9f9b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:23.662590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95534aec-8a0e-4f6b-9da5-8443936f9f9b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:23.712588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efd65b-937b-43d8-804a-a6bb3d3ccd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:23.713406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efd65b-937b-43d8-804a-a6bb3d3ccd79 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:23.713697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efd65b-937b-43d8-804a-a6bb3d3ccd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:23.713979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efd65b-937b-43d8-804a-a6bb3d3ccd79 None admin] PATH_INFO: `/v3/services/network` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:23.714664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efd65b-937b-43d8-804a-a6bb3d3ccd79 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:23.716035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efd65b-937b-43d8-804a-a6bb3d3ccd79 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-13efd65b-937b-43d8-804a-a6bb3d3ccd79 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:23.737040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:23.738106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:23.739374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 355/717] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:23 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:23.746479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74987df8-bdaa-4884-bed6-e4c0293f0922 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:23.968664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:24.037258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74987df8-bdaa-4884-bed6-e4c0293f0922 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:24.082843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8583a5-943b-4f09-a62b-f11b228f8f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:24.083512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8583a5-943b-4f09-a62b-f11b228f8f24 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:24.083805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8583a5-943b-4f09-a62b-f11b228f8f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:24.084086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8583a5-943b-4f09-a62b-f11b228f8f24 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:24.084855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8583a5-943b-4f09-a62b-f11b228f8f24 None admin] RBAC: Adding query filter params (name=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:24.085164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8583a5-943b-4f09-a62b-f11b228f8f24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:24.087079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8583a5-943b-4f09-a62b-f11b228f8f24 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:24.094509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 363/718] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:24.099684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efd65b-937b-43d8-804a-a6bb3d3ccd79 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:24.157733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f050-9aff-4f96-9e26-cd336780be67 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:24.158047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f050-9aff-4f96-9e26-cd336780be67 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:24.158300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05f050-9aff-4f96-9e26-cd336780be67 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:24.160146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 364/719] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:24.165107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c58e4-83a8-4e61-ae58-e51f6eb06d47 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:24.165395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c58e4-83a8-4e61-ae58-e51f6eb06d47 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:24.165674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c58e4-83a8-4e61-ae58-e51f6eb06d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:24.180094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:24.194625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6e3c58e4-83a8-4e61-ae58-e51f6eb06d47 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:24.195859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6e3c58e4-83a8-4e61-ae58-e51f6eb06d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:24.225800 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efd65b-937b-43d8-804a-a6bb3d3ccd79 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:24.275997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ce635-55c5-41ab-a01e-28a646684d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:24.278145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ce635-55c5-41ab-a01e-28a646684d78 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:24.278847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ce635-55c5-41ab-a01e-28a646684d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:24.280169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ce635-55c5-41ab-a01e-28a646684d78 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:24.281786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ce635-55c5-41ab-a01e-28a646684d78 None admin] RBAC: Adding query filter params (type=network) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:24.282475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ce635-55c5-41ab-a01e-28a646684d78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:24.285020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539ce635-55c5-41ab-a01e-28a646684d78 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:24.295039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 356/720] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:24.324065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 365/721] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 20:51:24.360338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ce635-55c5-41ab-a01e-28a646684d78 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:24.479086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ce635-55c5-41ab-a01e-28a646684d78 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:24.507744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:24.524414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5cb66-2622-4dda-a8b7-5d72c6df4a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:24.525840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5cb66-2622-4dda-a8b7-5d72c6df4a8e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:24.526565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5cb66-2622-4dda-a8b7-5d72c6df4a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:24.527111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5cb66-2622-4dda-a8b7-5d72c6df4a8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:24.543460 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5cb66-2622-4dda-a8b7-5d72c6df4a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ddc5cb66-2622-4dda-a8b7-5d72c6df4a8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:24.551326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:24.552312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:24.553909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 357/722] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:24.559255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c58e4-83a8-4e61-ae58-e51f6eb06d47 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:24.614831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:24.670022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c58e4-83a8-4e61-ae58-e51f6eb06d47 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:24.717962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c89d74-73f1-491a-bb15-9a1b89750a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:24.718651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c89d74-73f1-491a-bb15-9a1b89750a88 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:24.718937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c89d74-73f1-491a-bb15-9a1b89750a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:24.719214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c89d74-73f1-491a-bb15-9a1b89750a88 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:24.719963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c89d74-73f1-491a-bb15-9a1b89750a88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:24.720270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c89d74-73f1-491a-bb15-9a1b89750a88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:24.721639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c89d74-73f1-491a-bb15-9a1b89750a88 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:24.734872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 366/723] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:24.743604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5cb66-2622-4dda-a8b7-5d72c6df4a8e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:24.930788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5cb66-2622-4dda-a8b7-5d72c6df4a8e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:24.991463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d7df1-872c-41e4-a3fb-7a5bc49588d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:24.992720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d7df1-872c-41e4-a3fb-7a5bc49588d3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:24.993012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d7df1-872c-41e4-a3fb-7a5bc49588d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:24.993292 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d7df1-872c-41e4-a3fb-7a5bc49588d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.009072 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d7df1-872c-41e4-a3fb-7a5bc49588d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:25.010815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d7df1-872c-41e4-a3fb-7a5bc49588d3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1c5d7df1-872c-41e4-a3fb-7a5bc49588d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:25.033471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:25.034409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:25.035805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:25.035805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:25.035805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:25.035805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:25.035805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 358/724] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:25.040650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c89d74-73f1-491a-bb15-9a1b89750a88 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:25.166733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:25.177264 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c89d74-73f1-491a-bb15-9a1b89750a88 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:25.201105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab09565-96d6-4ca6-b598-b092d9d2b7e3 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.201464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab09565-96d6-4ca6-b598-b092d9d2b7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.201673 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab09565-96d6-4ca6-b598-b092d9d2b7e3 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.203316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 359/725] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:25.215584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6840d9-519a-40fc-bb73-63f65b90c1a4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.215862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6840d9-519a-40fc-bb73-63f65b90c1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.216142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6840d9-519a-40fc-bb73-63f65b90c1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.243132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9b6840d9-519a-40fc-bb73-63f65b90c1a4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:25.244270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9b6840d9-519a-40fc-bb73-63f65b90c1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:25.250384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c656c-e6b0-4965-8687-2d5e4ae5f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:25.250948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c656c-e6b0-4965-8687-2d5e4ae5f46d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.251231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c656c-e6b0-4965-8687-2d5e4ae5f46d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.251829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c656c-e6b0-4965-8687-2d5e4ae5f46d None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.252567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c656c-e6b0-4965-8687-2d5e4ae5f46d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:25.252874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c656c-e6b0-4965-8687-2d5e4ae5f46d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:25.254262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c656c-e6b0-4965-8687-2d5e4ae5f46d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:25.266738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 367/726] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:25.268060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f11f8a-0ebe-4997-a320-540d136e36c7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.268351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f11f8a-0ebe-4997-a320-540d136e36c7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.268632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f11f8a-0ebe-4997-a320-540d136e36c7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.277872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 368/727] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:25.277872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f11f8a-0ebe-4997-a320-540d136e36c7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:25.283716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:25.328871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 360/728] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:25.330478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d48c4-b792-446b-b8ee-daaba82ad3a0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.330955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d48c4-b792-446b-b8ee-daaba82ad3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.331415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d48c4-b792-446b-b8ee-daaba82ad3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.361278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cf4d48c4-b792-446b-b8ee-daaba82ad3a0 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:25.362800 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cf4d48c4-b792-446b-b8ee-daaba82ad3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:25.413165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f11f8a-0ebe-4997-a320-540d136e36c7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:25.442556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 361/729] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:25.443766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d48c4-b792-446b-b8ee-daaba82ad3a0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:25.461605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612cc96-1caa-4541-8737-a6028d7dc618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:25.462888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612cc96-1caa-4541-8737-a6028d7dc618 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.463412 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612cc96-1caa-4541-8737-a6028d7dc618 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.463889 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b612cc96-1caa-4541-8737-a6028d7dc618 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.472772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b612cc96-1caa-4541-8737-a6028d7dc618 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:25.474730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b612cc96-1caa-4541-8737-a6028d7dc618 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b612cc96-1caa-4541-8737-a6028d7dc618 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:25.491285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:25.492868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:25.493868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:25.493868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:25.493868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:25.493868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:25.493868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:25.493868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:25.493868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:25.493868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:25.493868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:25.493868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:25.496511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 369/730] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:25.497560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612cc96-1caa-4541-8737-a6028d7dc618 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:25.516040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:25.583965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:25.592488 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d48c4-b792-446b-b8ee-daaba82ad3a0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:25.618524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b612cc96-1caa-4541-8737-a6028d7dc618 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:25.636411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6e4dc-5848-4c54-90fe-22941923c012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:25.637269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6e4dc-5848-4c54-90fe-22941923c012 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.637550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6e4dc-5848-4c54-90fe-22941923c012 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.637833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6e4dc-5848-4c54-90fe-22941923c012 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.651842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b6e4dc-5848-4c54-90fe-22941923c012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:25.657347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d65ea0-dce4-4add-b627-a50f8af7a394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:25.658391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d65ea0-dce4-4add-b627-a50f8af7a394 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.658907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d65ea0-dce4-4add-b627-a50f8af7a394 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.659402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d65ea0-dce4-4add-b627-a50f8af7a394 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-20b6e4dc-5848-4c54-90fe-22941923c012 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:25.660334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:25.661258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:25.662104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:25.666651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 362/731] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:25.668419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6e4dc-5848-4c54-90fe-22941923c012 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:25.677578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d65ea0-dce4-4add-b627-a50f8af7a394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c2d65ea0-dce4-4add-b627-a50f8af7a394 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:25.685899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:25.686968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 370/732] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:25.693740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d65ea0-dce4-4add-b627-a50f8af7a394 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:25.786104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:25.797115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6e4dc-5848-4c54-90fe-22941923c012 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:25.822270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d65ea0-dce4-4add-b627-a50f8af7a394 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:25.837934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223f2bb-4e6e-4972-90dc-b6f66c0f7679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:25.838972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223f2bb-4e6e-4972-90dc-b6f66c0f7679 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.839524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223f2bb-4e6e-4972-90dc-b6f66c0f7679 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.839978 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223f2bb-4e6e-4972-90dc-b6f66c0f7679 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.840982 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223f2bb-4e6e-4972-90dc-b6f66c0f7679 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:25.841492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223f2bb-4e6e-4972-90dc-b6f66c0f7679 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:25.843234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223f2bb-4e6e-4972-90dc-b6f66c0f7679 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:25.856368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d3b8e-6ba1-4c13-bc3f-ff7abfae0f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:25.857026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d3b8e-6ba1-4c13-bc3f-ff7abfae0f3b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.857310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d3b8e-6ba1-4c13-bc3f-ff7abfae0f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.857588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d3b8e-6ba1-4c13-bc3f-ff7abfae0f3b None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.858325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d3b8e-6ba1-4c13-bc3f-ff7abfae0f3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:25.858663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d3b8e-6ba1-4c13-bc3f-ff7abfae0f3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:25.860033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d3b8e-6ba1-4c13-bc3f-ff7abfae0f3b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:25.866597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 363/733] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 26 20:51:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:25.867490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223f2bb-4e6e-4972-90dc-b6f66c0f7679 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:25.872818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 371/734] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:25.873569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05d4b-6f36-4527-9bc0-d24a33def031 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:25.873854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05d4b-6f36-4527-9bc0-d24a33def031 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:25.874131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd05d4b-6f36-4527-9bc0-d24a33def031 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:25.879471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 372/735] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:25.880266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05d4b-6f36-4527-9bc0-d24a33def031 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:25.886098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:25.973545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223f2bb-4e6e-4972-90dc-b6f66c0f7679 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:25.991907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd05d4b-6f36-4527-9bc0-d24a33def031 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:26.006163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a1c56-ef93-406d-9d1a-ee632d393dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:26.006852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a1c56-ef93-406d-9d1a-ee632d393dca None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.007144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a1c56-ef93-406d-9d1a-ee632d393dca None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.007941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a1c56-ef93-406d-9d1a-ee632d393dca None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.008684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a1c56-ef93-406d-9d1a-ee632d393dca None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:26.008990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a1c56-ef93-406d-9d1a-ee632d393dca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:26.010358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a1c56-ef93-406d-9d1a-ee632d393dca None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:26.022667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 364/736] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:26.022667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a1c56-ef93-406d-9d1a-ee632d393dca None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:26.034189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8240a1-80bc-4d79-b3d2-675f02fa3871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:26.034974 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8240a1-80bc-4d79-b3d2-675f02fa3871 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.035452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8240a1-80bc-4d79-b3d2-675f02fa3871 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.035792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8240a1-80bc-4d79-b3d2-675f02fa3871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.039670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:26.099621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:26.110636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8240a1-80bc-4d79-b3d2-675f02fa3871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:26.112884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8240a1-80bc-4d79-b3d2-675f02fa3871 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:26.124173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a1c56-ef93-406d-9d1a-ee632d393dca None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:26.160955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568ff9-f07b-4548-a785-34ff1ded0205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:26.161754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568ff9-f07b-4548-a785-34ff1ded0205 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.162035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568ff9-f07b-4548-a785-34ff1ded0205 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.162311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568ff9-f07b-4548-a785-34ff1ded0205 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.172968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568ff9-f07b-4548-a785-34ff1ded0205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:26.174399 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568ff9-f07b-4548-a785-34ff1ded0205 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ee568ff9-f07b-4548-a785-34ff1ded0205 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:26.191801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:26.192737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:26.193668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:26.193668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:26.193668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:26.193668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:26.193668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 365/737] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:26.194830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662ef7e-7fbb-4339-841e-e6ab19211b21 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.194830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662ef7e-7fbb-4339-841e-e6ab19211b21 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.194830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662ef7e-7fbb-4339-841e-e6ab19211b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.212667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0662ef7e-7fbb-4339-841e-e6ab19211b21 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:26.213841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0662ef7e-7fbb-4339-841e-e6ab19211b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:26.227355 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 373/738] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 20:51:26.232611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8240a1-80bc-4d79-b3d2-675f02fa3871 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:26.271918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:26.286301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 366/739] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:26.287190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662ef7e-7fbb-4339-841e-e6ab19211b21 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:26.348320 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8240a1-80bc-4d79-b3d2-675f02fa3871 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:26.354168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:26.384578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1749b0-865e-41d6-9887-0ed175c77fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:26.385795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1749b0-865e-41d6-9887-0ed175c77fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.386176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1749b0-865e-41d6-9887-0ed175c77fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.386580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1749b0-865e-41d6-9887-0ed175c77fe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.396688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1749b0-865e-41d6-9887-0ed175c77fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:26.397153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662ef7e-7fbb-4339-841e-e6ab19211b21 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:26.398837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1749b0-865e-41d6-9887-0ed175c77fe4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7e1749b0-865e-41d6-9887-0ed175c77fe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:26.417312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:26.418516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:26.419651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:26.419651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:26.419651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:26.419651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:26.420189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 374/740] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:26.420964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1749b0-865e-41d6-9887-0ed175c77fe4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:26.438401 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604d94a-fafd-4bad-95c5-b32c79452aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:26.439095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604d94a-fafd-4bad-95c5-b32c79452aad None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.439374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604d94a-fafd-4bad-95c5-b32c79452aad None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.439643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604d94a-fafd-4bad-95c5-b32c79452aad None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.440362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604d94a-fafd-4bad-95c5-b32c79452aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:26.440665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604d94a-fafd-4bad-95c5-b32c79452aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:26.442025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604d94a-fafd-4bad-95c5-b32c79452aad None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:26.457276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 367/741] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:26.457276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604d94a-fafd-4bad-95c5-b32c79452aad None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:26.516424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:26.529663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1749b0-865e-41d6-9887-0ed175c77fe4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:26.565639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d0bac-8d1f-43fd-9396-19295543dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:26.566431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d0bac-8d1f-43fd-9396-19295543dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.566765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d0bac-8d1f-43fd-9396-19295543dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.567042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d0bac-8d1f-43fd-9396-19295543dfa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.567434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604d94a-fafd-4bad-95c5-b32c79452aad None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:26.577186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d0bac-8d1f-43fd-9396-19295543dfa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:26.578707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d0bac-8d1f-43fd-9396-19295543dfa0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:26.586616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-285d0bac-8d1f-43fd-9396-19295543dfa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:26.597206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:26.598189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:26.599161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:26.599161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:26.599161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:26.599161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:26.599161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 375/742] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:26.599487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d0bac-8d1f-43fd-9396-19295543dfa0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:26.613379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625f244-8c53-4ca9-aa8e-1d1c4b57e194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:26.614078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625f244-8c53-4ca9-aa8e-1d1c4b57e194 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.614364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625f244-8c53-4ca9-aa8e-1d1c4b57e194 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.614679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625f244-8c53-4ca9-aa8e-1d1c4b57e194 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.615412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d625f244-8c53-4ca9-aa8e-1d1c4b57e194 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:26.616879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d625f244-8c53-4ca9-aa8e-1d1c4b57e194 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:26.642484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 368/743] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:51:26 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:51:26.642484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625f244-8c53-4ca9-aa8e-1d1c4b57e194 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:26.739790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d0bac-8d1f-43fd-9396-19295543dfa0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:26.792384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f4d88-eec3-4dcd-a4be-5368cc8e0964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:26.793174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f4d88-eec3-4dcd-a4be-5368cc8e0964 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.793469 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f4d88-eec3-4dcd-a4be-5368cc8e0964 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.793751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f4d88-eec3-4dcd-a4be-5368cc8e0964 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.794516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f4d88-eec3-4dcd-a4be-5368cc8e0964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:26.794832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f4d88-eec3-4dcd-a4be-5368cc8e0964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:26.795954 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625f244-8c53-4ca9-aa8e-1d1c4b57e194 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:26.796381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f4d88-eec3-4dcd-a4be-5368cc8e0964 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:26.799382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:26.811864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 376/744] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:26.813385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f4d88-eec3-4dcd-a4be-5368cc8e0964 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:26.837985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d5f06-6e40-462e-96e0-11b181f46117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:26.841440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d5f06-6e40-462e-96e0-11b181f46117 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.841440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d5f06-6e40-462e-96e0-11b181f46117 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.841440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d5f06-6e40-462e-96e0-11b181f46117 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.847963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d5f06-6e40-462e-96e0-11b181f46117 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:26.849512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d5f06-6e40-462e-96e0-11b181f46117 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3a5d5f06-6e40-462e-96e0-11b181f46117 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:26.864414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:26.865400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:26.866462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 369/745] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 26 20:51:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:26.867119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e87a5-f698-406b-a999-5ae9dea0393b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.867195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e87a5-f698-406b-a999-5ae9dea0393b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.867466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e87a5-f698-406b-a999-5ae9dea0393b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.868702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 370/746] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:26.869446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e87a5-f698-406b-a999-5ae9dea0393b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:26.905368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:26.935061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f4d88-eec3-4dcd-a4be-5368cc8e0964 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:26.971598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2216104-91f9-4dce-b2f2-a8f582047c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:26.972266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2216104-91f9-4dce-b2f2-a8f582047c38 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:26.972557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2216104-91f9-4dce-b2f2-a8f582047c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:26.972838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2216104-91f9-4dce-b2f2-a8f582047c38 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:26.973569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2216104-91f9-4dce-b2f2-a8f582047c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:26.973899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2216104-91f9-4dce-b2f2-a8f582047c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:26.975317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2216104-91f9-4dce-b2f2-a8f582047c38 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:26.983622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e87a5-f698-406b-a999-5ae9dea0393b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:26.987027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 377/747] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:26.987838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2216104-91f9-4dce-b2f2-a8f582047c38 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:27.020769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7e8a0-5a29-494c-8619-338c55027494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:27.021582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7e8a0-5a29-494c-8619-338c55027494 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:27.021880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7e8a0-5a29-494c-8619-338c55027494 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:27.022157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7e8a0-5a29-494c-8619-338c55027494 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:27.030008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7e8a0-5a29-494c-8619-338c55027494 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:27.031502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7e8a0-5a29-494c-8619-338c55027494 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c2c7e8a0-5a29-494c-8619-338c55027494 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:27.046554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:27.047497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:27.048440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 371/748] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 26 20:51:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:27.049397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302ade3-043c-4d3a-b179-ece47ce0b0bc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:27.049397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302ade3-043c-4d3a-b179-ece47ce0b0bc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:27.049528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302ade3-043c-4d3a-b179-ece47ce0b0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:27.053668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:27.066578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2302ade3-043c-4d3a-b179-ece47ce0b0bc None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:27.067772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2302ade3-043c-4d3a-b179-ece47ce0b0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:27.100078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2216104-91f9-4dce-b2f2-a8f582047c38 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:27.134395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 372/749] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 20:51:27.135266 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302ade3-043c-4d3a-b179-ece47ce0b0bc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:27.143371 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92256c92-2f11-455b-956b-553f1fa8370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:27.144335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92256c92-2f11-455b-956b-553f1fa8370b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:27.144694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92256c92-2f11-455b-956b-553f1fa8370b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:27.145043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92256c92-2f11-455b-956b-553f1fa8370b None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:27.145956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92256c92-2f11-455b-956b-553f1fa8370b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:27.146336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92256c92-2f11-455b-956b-553f1fa8370b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:27.147882 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92256c92-2f11-455b-956b-553f1fa8370b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:27.166536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:27.171242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 378/750] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 26 20:51:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:27.172872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92256c92-2f11-455b-956b-553f1fa8370b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:27.244000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302ade3-043c-4d3a-b179-ece47ce0b0bc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:27.279006 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eee57-929e-4fbb-85fd-368a0b7c3e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:27.279808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eee57-929e-4fbb-85fd-368a0b7c3e4b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:27.280097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eee57-929e-4fbb-85fd-368a0b7c3e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:27.280381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eee57-929e-4fbb-85fd-368a0b7c3e4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:27.284275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92256c92-2f11-455b-956b-553f1fa8370b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:27.287871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446eee57-929e-4fbb-85fd-368a0b7c3e4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:27.289363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446eee57-929e-4fbb-85fd-368a0b7c3e4b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:27.298355 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-446eee57-929e-4fbb-85fd-368a0b7c3e4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:27.305181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:27.306104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:27.307041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:27.307041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:27.307041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:27.307041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:27.307041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:27.307041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:27.307041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:27.307041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:27.307041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 373/751] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:27.307534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eee57-929e-4fbb-85fd-368a0b7c3e4b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:27.328857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec77c1-49d4-43fd-ab1c-496120d5d676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:27.329530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec77c1-49d4-43fd-ab1c-496120d5d676 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:27.329813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec77c1-49d4-43fd-ab1c-496120d5d676 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:27.330090 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec77c1-49d4-43fd-ab1c-496120d5d676 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:27.330970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec77c1-49d4-43fd-ab1c-496120d5d676 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:27.331280 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec77c1-49d4-43fd-ab1c-496120d5d676 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:27.332714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec77c1-49d4-43fd-ab1c-496120d5d676 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:27.350912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 379/752] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 26 20:51:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:27.355958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec77c1-49d4-43fd-ab1c-496120d5d676 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:27.399842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:27.418473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eee57-929e-4fbb-85fd-368a0b7c3e4b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:27.453218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd8c1a-c62f-476d-9161-eded72406b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:27.454110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd8c1a-c62f-476d-9161-eded72406b98 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:27.454575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd8c1a-c62f-476d-9161-eded72406b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:27.455012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd8c1a-c62f-476d-9161-eded72406b98 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:27.455866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd8c1a-c62f-476d-9161-eded72406b98 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:27.457382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd8c1a-c62f-476d-9161-eded72406b98 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:27.462731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec77c1-49d4-43fd-ab1c-496120d5d676 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:27.471907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 374/753] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:51:27 2022] POST /identity/v3/services => generated 240 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:51:27.473117 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd8c1a-c62f-476d-9161-eded72406b98 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:27.518884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e11d8-02c3-4896-800f-6de23548935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:27.519695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e11d8-02c3-4896-800f-6de23548935a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:27.519990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e11d8-02c3-4896-800f-6de23548935a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:27.520264 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e11d8-02c3-4896-800f-6de23548935a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:27.532563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e11d8-02c3-4896-800f-6de23548935a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:27.534546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e11d8-02c3-4896-800f-6de23548935a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e03e11d8-02c3-4896-800f-6de23548935a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:27.554239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:27.555614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:27.556980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:27.556980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:27.556980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:27.556980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:27.556980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 380/754] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:27.557492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e11d8-02c3-4896-800f-6de23548935a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:27.585227 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:27.608213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd8c1a-c62f-476d-9161-eded72406b98 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:27.657528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bdd79-c131-4f95-83db-7d612e4aadbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:27.658309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bdd79-c131-4f95-83db-7d612e4aadbd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:27.658699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bdd79-c131-4f95-83db-7d612e4aadbd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:27.659053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bdd79-c131-4f95-83db-7d612e4aadbd None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:27.660074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bdd79-c131-4f95-83db-7d612e4aadbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:27.660460 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bdd79-c131-4f95-83db-7d612e4aadbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:27.662115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bdd79-c131-4f95-83db-7d612e4aadbd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:27.682028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e11d8-02c3-4896-800f-6de23548935a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:27.682225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 375/755] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:27.683287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bdd79-c131-4f95-83db-7d612e4aadbd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:27.687773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:27.725049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7597d9-ca2a-49f2-a57a-173383f20a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:27.725704 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7597d9-ca2a-49f2-a57a-173383f20a2d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:27.725994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7597d9-ca2a-49f2-a57a-173383f20a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:27.726312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7597d9-ca2a-49f2-a57a-173383f20a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:27.810970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bdd79-c131-4f95-83db-7d612e4aadbd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:27.815838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7597d9-ca2a-49f2-a57a-173383f20a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:27.817380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7597d9-ca2a-49f2-a57a-173383f20a2d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:27.842390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:27.855430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ac18a-b2a6-47aa-bade-12231c30ccf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:27.856886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ac18a-b2a6-47aa-bade-12231c30ccf5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:27.857181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ac18a-b2a6-47aa-bade-12231c30ccf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:27.857464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ac18a-b2a6-47aa-bade-12231c30ccf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:27.858200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ac18a-b2a6-47aa-bade-12231c30ccf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:27.858601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ac18a-b2a6-47aa-bade-12231c30ccf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:27.859928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ac18a-b2a6-47aa-bade-12231c30ccf5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:27.878517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 376/756] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:27.879678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ac18a-b2a6-47aa-bade-12231c30ccf5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:27.931292 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 381/757] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:51:27.932515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7597d9-ca2a-49f2-a57a-173383f20a2d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:27.969609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:28.027240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ac18a-b2a6-47aa-bade-12231c30ccf5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:28.053943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7597d9-ca2a-49f2-a57a-173383f20a2d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:28.076061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:28.078652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbaec79-e109-4540-8a2c-ff2e60924025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:28.079484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbaec79-e109-4540-8a2c-ff2e60924025 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:28.080054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbaec79-e109-4540-8a2c-ff2e60924025 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:28.080345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbaec79-e109-4540-8a2c-ff2e60924025 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:28.081494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbaec79-e109-4540-8a2c-ff2e60924025 None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=1285406084604bd19287de9e1beeeaa8) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:28.081826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbaec79-e109-4540-8a2c-ff2e60924025 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:28.084015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbaec79-e109-4540-8a2c-ff2e60924025 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:28.093000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 377/758] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:27 2022] GET /identity/v3/role_assignments?user.id=1285406084604bd19287de9e1beeeaa8&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:28.093234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cbeeb-76d4-45d9-8f9e-a90727816ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:28.093816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbaec79-e109-4540-8a2c-ff2e60924025 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:28.094250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cbeeb-76d4-45d9-8f9e-a90727816ead None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:28.094693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cbeeb-76d4-45d9-8f9e-a90727816ead None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:28.095080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cbeeb-76d4-45d9-8f9e-a90727816ead None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:28.102702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cbeeb-76d4-45d9-8f9e-a90727816ead None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:28.104373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cbeeb-76d4-45d9-8f9e-a90727816ead None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4e9cbeeb-76d4-45d9-8f9e-a90727816ead None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:28.120236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:28.121379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:28.122535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:28.122535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:28.122535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:28.122535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:28.122535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:28.122535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:28.122535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:28.122535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:28.124345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 382/759] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:28.131260 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cbeeb-76d4-45d9-8f9e-a90727816ead None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:28.238931 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbaec79-e109-4540-8a2c-ff2e60924025 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:28.295587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cbeeb-76d4-45d9-8f9e-a90727816ead None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:28.298801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:28.327188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d7b87-1629-4b2a-af4b-7e4853fa8056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:28.328170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d7b87-1629-4b2a-af4b-7e4853fa8056 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:28.328558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d7b87-1629-4b2a-af4b-7e4853fa8056 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:28.328940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41d7b87-1629-4b2a-af4b-7e4853fa8056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:28.343826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f514eaf-4f4d-46c9-90c0-5207c7086fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:28.344717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f514eaf-4f4d-46c9-90c0-5207c7086fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:28.345108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f514eaf-4f4d-46c9-90c0-5207c7086fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:28.345489 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f514eaf-4f4d-46c9-90c0-5207c7086fc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:28.346396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f514eaf-4f4d-46c9-90c0-5207c7086fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:28.346832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f514eaf-4f4d-46c9-90c0-5207c7086fc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:28.346973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d7b87-1629-4b2a-af4b-7e4853fa8056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:28.348414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f514eaf-4f4d-46c9-90c0-5207c7086fc9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:28.348531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41d7b87-1629-4b2a-af4b-7e4853fa8056 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:28.367627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 383/760] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f41d7b87-1629-4b2a-af4b-7e4853fa8056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:28.368433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:28.369508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:28.370724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:28.370724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:28.370724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:28.370724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:28.373595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 378/761] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:28.373728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f514eaf-4f4d-46c9-90c0-5207c7086fc9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:28.375382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d7b87-1629-4b2a-af4b-7e4853fa8056 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:28.428899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:28.495983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41d7b87-1629-4b2a-af4b-7e4853fa8056 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:28.532399 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937febff-fde4-4e8c-885f-4d3128519e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:28.533067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937febff-fde4-4e8c-885f-4d3128519e29 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:28.533350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937febff-fde4-4e8c-885f-4d3128519e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:28.533627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937febff-fde4-4e8c-885f-4d3128519e29 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:28.534406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937febff-fde4-4e8c-885f-4d3128519e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:28.534750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937febff-fde4-4e8c-885f-4d3128519e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:28.536245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937febff-fde4-4e8c-885f-4d3128519e29 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:28.553069 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 379/762] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:28.555319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937febff-fde4-4e8c-885f-4d3128519e29 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:28.569696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f514eaf-4f4d-46c9-90c0-5207c7086fc9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:28.614869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:28.629599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd34947-04a1-4841-a8cd-e04e2d0f7d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:28.630435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd34947-04a1-4841-a8cd-e04e2d0f7d5c None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:28.630755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd34947-04a1-4841-a8cd-e04e2d0f7d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:28.631412 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd34947-04a1-4841-a8cd-e04e2d0f7d5c None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/490d773bf21446789b77c5793a518a5a/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:28.633920 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd34947-04a1-4841-a8cd-e04e2d0f7d5c None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=490d773bf21446789b77c5793a518a5a, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:28.635742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd34947-04a1-4841-a8cd-e04e2d0f7d5c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:28.657043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 384/763] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:51:28 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/490d773bf21446789b77c5793a518a5a/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:51:28.701413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937febff-fde4-4e8c-885f-4d3128519e29 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:28.755936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1a60-5242-41ad-804f-e6bb9f04bb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:28.756780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1a60-5242-41ad-804f-e6bb9f04bb91 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:28.757071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1a60-5242-41ad-804f-e6bb9f04bb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:28.757999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1a60-5242-41ad-804f-e6bb9f04bb91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:28.768423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb1a60-5242-41ad-804f-e6bb9f04bb91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:28.769883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb1a60-5242-41ad-804f-e6bb9f04bb91 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-fcdb1a60-5242-41ad-804f-e6bb9f04bb91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:28.792700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:28.793703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:28.794634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:28.794634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:28.794634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:28.794634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:28.794634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:28.794634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:28.794634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:28.794634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:28.798746 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 380/764] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:28.803445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd34947-04a1-4841-a8cd-e04e2d0f7d5c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:28.969693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:28.972633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd34947-04a1-4841-a8cd-e04e2d0f7d5c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:29.010264 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafbf68-6bcc-4096-b166-18399006ffae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:29.011214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafbf68-6bcc-4096-b166-18399006ffae None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:29.011498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafbf68-6bcc-4096-b166-18399006ffae None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:29.011773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafbf68-6bcc-4096-b166-18399006ffae None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:29.012559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aafbf68-6bcc-4096-b166-18399006ffae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:29.012870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aafbf68-6bcc-4096-b166-18399006ffae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:29.014318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aafbf68-6bcc-4096-b166-18399006ffae None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:29.027141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 385/765] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:29.033100 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1a60-5242-41ad-804f-e6bb9f04bb91 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:29.066484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f586109-9803-4af2-9dae-6df08d98ab92 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:29.066828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f586109-9803-4af2-9dae-6df08d98ab92 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:29.067067 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f586109-9803-4af2-9dae-6df08d98ab92 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:29.068694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 386/766] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:29.073851 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11e89a-aa8d-40d8-a0c6-e7b020fe5e70 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:29.074140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11e89a-aa8d-40d8-a0c6-e7b020fe5e70 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:29.074420 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11e89a-aa8d-40d8-a0c6-e7b020fe5e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:29.102896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5c11e89a-aa8d-40d8-a0c6-e7b020fe5e70 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:29.104085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5c11e89a-aa8d-40d8-a0c6-e7b020fe5e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:29.156803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:29.160477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1a60-5242-41ad-804f-e6bb9f04bb91 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:29.227162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fab577-a36e-478e-b831-590c398574d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:29.227725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fab577-a36e-478e-b831-590c398574d2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:29.228008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fab577-a36e-478e-b831-590c398574d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:29.228300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fab577-a36e-478e-b831-590c398574d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:29.229238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fab577-a36e-478e-b831-590c398574d2 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=e67f497e8a8b46e1b665999ba8fda38d) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:29.229557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fab577-a36e-478e-b831-590c398574d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:29.231802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fab577-a36e-478e-b831-590c398574d2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:29.244781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 387/768] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:29.244995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 381/768] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:29 2022] GET /identity/v3/role_assignments?user.id=e67f497e8a8b46e1b665999ba8fda38d&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:29.279348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fab577-a36e-478e-b831-590c398574d2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:29.402271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fab577-a36e-478e-b831-590c398574d2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:29.449307 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed58661-c842-49ef-81ca-579e0b051005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:29.450177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed58661-c842-49ef-81ca-579e0b051005 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:29.450481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed58661-c842-49ef-81ca-579e0b051005 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:29.450763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed58661-c842-49ef-81ca-579e0b051005 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:29.451425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed58661-c842-49ef-81ca-579e0b051005 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:29.452805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed58661-c842-49ef-81ca-579e0b051005 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-aed58661-c842-49ef-81ca-579e0b051005 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:29.472101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:29.473136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:29.474008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 382/769] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 26 20:51:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:29.480101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11e89a-aa8d-40d8-a0c6-e7b020fe5e70 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:29.611131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:29.622075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11e89a-aa8d-40d8-a0c6-e7b020fe5e70 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:29.665126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c33d0f-c893-4a32-84fb-5fa34a89dcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:29.665802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c33d0f-c893-4a32-84fb-5fa34a89dcee None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:29.666088 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c33d0f-c893-4a32-84fb-5fa34a89dcee None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:29.666365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c33d0f-c893-4a32-84fb-5fa34a89dcee None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:29.667211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c33d0f-c893-4a32-84fb-5fa34a89dcee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:29.667511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c33d0f-c893-4a32-84fb-5fa34a89dcee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:29.668900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c33d0f-c893-4a32-84fb-5fa34a89dcee None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:29.674167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 388/770] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 26 20:51:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:29.679902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed58661-c842-49ef-81ca-579e0b051005 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:29.717048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:29.816682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed58661-c842-49ef-81ca-579e0b051005 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:29.868565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fca53-e25a-4d5b-9cc5-7ca258191fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:29.869240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fca53-e25a-4d5b-9cc5-7ca258191fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:29.869526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fca53-e25a-4d5b-9cc5-7ca258191fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:29.869810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fca53-e25a-4d5b-9cc5-7ca258191fd1 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:29.870549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fca53-e25a-4d5b-9cc5-7ca258191fd1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:29.870927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fca53-e25a-4d5b-9cc5-7ca258191fd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:29.872371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fca53-e25a-4d5b-9cc5-7ca258191fd1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:29.878659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 383/771] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 26 20:51:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:29.885715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c33d0f-c893-4a32-84fb-5fa34a89dcee None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:29.948388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19423b-4ed3-4d49-9af9-2c61fe8aa9b8 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:29.948715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19423b-4ed3-4d49-9af9-2c61fe8aa9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:29.948960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19423b-4ed3-4d49-9af9-2c61fe8aa9b8 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:29.950616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 384/772] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:29.955317 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c224f6-6fca-4fec-b9bc-6b432d32ec28 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:29.955611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c224f6-6fca-4fec-b9bc-6b432d32ec28 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:29.955899 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c224f6-6fca-4fec-b9bc-6b432d32ec28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:29.992558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e0c224f6-6fca-4fec-b9bc-6b432d32ec28 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:29.993770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e0c224f6-6fca-4fec-b9bc-6b432d32ec28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:30.023994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c33d0f-c893-4a32-84fb-5fa34a89dcee None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:30.066653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c29f6-a791-4b76-af40-33f5d86d9859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:30.067389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c29f6-a791-4b76-af40-33f5d86d9859 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:30.067682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c29f6-a791-4b76-af40-33f5d86d9859 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:30.067963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c29f6-a791-4b76-af40-33f5d86d9859 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:30.068764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c29f6-a791-4b76-af40-33f5d86d9859 None admin] RBAC: Adding query filter params (interface=public, service_id=c5dc9bac56184012acd40db12ab30bc0, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:30.069077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c29f6-a791-4b76-af40-33f5d86d9859 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:30.071380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c29f6-a791-4b76-af40-33f5d86d9859 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:30.085739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 389/773] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 26 20:51:29 2022] GET /identity/v3/endpoints?service_id=c5dc9bac56184012acd40db12ab30bc0&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:30.090697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c29f6-a791-4b76-af40-33f5d86d9859 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:30.099977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 385/774] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:30.124791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:30.127576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c224f6-6fca-4fec-b9bc-6b432d32ec28 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:30.153764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:30.305186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c29f6-a791-4b76-af40-33f5d86d9859 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:30.386526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290ef4f-2286-4e32-b04c-e0918d1ec8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:30.387656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290ef4f-2286-4e32-b04c-e0918d1ec8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:30.388147 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290ef4f-2286-4e32-b04c-e0918d1ec8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:30.388616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290ef4f-2286-4e32-b04c-e0918d1ec8e3 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:30.389490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290ef4f-2286-4e32-b04c-e0918d1ec8e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:30.391240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290ef4f-2286-4e32-b04c-e0918d1ec8e3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:30.397480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 390/775] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 26 20:51:30 2022] GET /identity/v3/services => generated 772 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:30.454744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c224f6-6fca-4fec-b9bc-6b432d32ec28 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:30.514575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c8150-347c-427e-91e8-664c041b7025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:30.515775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c8150-347c-427e-91e8-664c041b7025 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:30.516304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c8150-347c-427e-91e8-664c041b7025 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:30.516779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c8150-347c-427e-91e8-664c041b7025 None admin] PATH_INFO: `/v3/services/network` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:30.517641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c8150-347c-427e-91e8-664c041b7025 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:30.519322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c8150-347c-427e-91e8-664c041b7025 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7b6c8150-347c-427e-91e8-664c041b7025 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:30.534321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:30.535405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:51:30.536261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:30.544439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 386/776] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:30 2022] GET /identity/v3/services/network => generated 88 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:30.546172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290ef4f-2286-4e32-b04c-e0918d1ec8e3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:30.646822 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:30.667788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290ef4f-2286-4e32-b04c-e0918d1ec8e3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:30.703238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a4bbc-2566-44ba-a784-7caffb0ac156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:30.704239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a4bbc-2566-44ba-a784-7caffb0ac156 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:30.704739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a4bbc-2566-44ba-a784-7caffb0ac156 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:30.705209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a4bbc-2566-44ba-a784-7caffb0ac156 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:30.706116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a4bbc-2566-44ba-a784-7caffb0ac156 None admin] RBAC: Adding query filter params (name=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:30.706662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a4bbc-2566-44ba-a784-7caffb0ac156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:30.708232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6a4bbc-2566-44ba-a784-7caffb0ac156 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:30.713479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 391/777] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:30.719355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c8150-347c-427e-91e8-664c041b7025 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:30.823214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:30.840342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c8150-347c-427e-91e8-664c041b7025 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:30.875751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db969384-44d9-4114-8db7-bf052147ec76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:30.876431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db969384-44d9-4114-8db7-bf052147ec76 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:30.876713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db969384-44d9-4114-8db7-bf052147ec76 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:30.876986 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db969384-44d9-4114-8db7-bf052147ec76 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:30.877710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db969384-44d9-4114-8db7-bf052147ec76 None admin] RBAC: Adding query filter params (type=network) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:30.878018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db969384-44d9-4114-8db7-bf052147ec76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:30.879621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db969384-44d9-4114-8db7-bf052147ec76 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:30.886063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 387/778] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:30.891970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a4bbc-2566-44ba-a784-7caffb0ac156 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:30.959045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c03e1-8377-4b3a-8011-6ba01f718778 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:30.959346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c03e1-8377-4b3a-8011-6ba01f718778 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:30.959536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c03e1-8377-4b3a-8011-6ba01f718778 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:30.961696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 388/779] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:30.966666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b9f4c-f7fd-4941-bb35-24286e13882a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:30.966956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b9f4c-f7fd-4941-bb35-24286e13882a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:30.967242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b9f4c-f7fd-4941-bb35-24286e13882a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:30.996451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-062b9f4c-f7fd-4941-bb35-24286e13882a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:30.997656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-062b9f4c-f7fd-4941-bb35-24286e13882a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:30.999832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6a4bbc-2566-44ba-a784-7caffb0ac156 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:31.035961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbe747-a6a7-4bf5-a4f4-cbac13658876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:31.036721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbe747-a6a7-4bf5-a4f4-cbac13658876 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:31.036888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbe747-a6a7-4bf5-a4f4-cbac13658876 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:31.037170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbe747-a6a7-4bf5-a4f4-cbac13658876 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:31.037957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbe747-a6a7-4bf5-a4f4-cbac13658876 None admin] RBAC: Adding query filter params (interface=public, service_id=2260e19f44f04d81945c537f2c1c17c2, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:31.038267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbe747-a6a7-4bf5-a4f4-cbac13658876 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:31.040431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbe747-a6a7-4bf5-a4f4-cbac13658876 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:31.046294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 392/780] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 26 20:51:30 2022] GET /identity/v3/endpoints?service_id=2260e19f44f04d81945c537f2c1c17c2&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:31.051535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbe747-a6a7-4bf5-a4f4-cbac13658876 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:31.064361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:31.152193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 389/781] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:31.186232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b9f4c-f7fd-4941-bb35-24286e13882a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:31.207482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbe747-a6a7-4bf5-a4f4-cbac13658876 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:31.275368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45262334-32f8-48ae-9909-5626004b4ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:31.276048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45262334-32f8-48ae-9909-5626004b4ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:31.276338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45262334-32f8-48ae-9909-5626004b4ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:31.276617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45262334-32f8-48ae-9909-5626004b4ec8 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:31.277321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45262334-32f8-48ae-9909-5626004b4ec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:31.278780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45262334-32f8-48ae-9909-5626004b4ec8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:31.291033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 393/782] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 26 20:51:31 2022] GET /identity/v3/services => generated 772 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:31.303528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:31.328460 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b9f4c-f7fd-4941-bb35-24286e13882a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:31.374247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92127d-72ff-43ae-93eb-5082af37e084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:31.375103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92127d-72ff-43ae-93eb-5082af37e084 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:31.375385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92127d-72ff-43ae-93eb-5082af37e084 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:31.375663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92127d-72ff-43ae-93eb-5082af37e084 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:31.393017 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92127d-72ff-43ae-93eb-5082af37e084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-bf92127d-72ff-43ae-93eb-5082af37e084 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:31.401501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:31.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:31.403416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:31.410569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 390/783] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:31.411637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45262334-32f8-48ae-9909-5626004b4ec8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:31.472740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa558a-eed5-46a7-87e6-6bad2910fb85 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:31.473029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa558a-eed5-46a7-87e6-6bad2910fb85 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:31.473312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa558a-eed5-46a7-87e6-6bad2910fb85 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:31.474720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 391/784] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:31.480367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68ca28-b26d-40ef-8bb2-d4d84c96e7f0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:31.480651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68ca28-b26d-40ef-8bb2-d4d84c96e7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:31.480947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68ca28-b26d-40ef-8bb2-d4d84c96e7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:31.512715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5a68ca28-b26d-40ef-8bb2-d4d84c96e7f0 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:31.514075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5a68ca28-b26d-40ef-8bb2-d4d84c96e7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:31.546858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:31.578813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45262334-32f8-48ae-9909-5626004b4ec8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:31.619499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 392/785] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:31.626493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001baeec-2f5e-4b8f-afee-8ecca21e743f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:31.626493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001baeec-2f5e-4b8f-afee-8ecca21e743f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:31.626493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001baeec-2f5e-4b8f-afee-8ecca21e743f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:31.626493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001baeec-2f5e-4b8f-afee-8ecca21e743f None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:31.626856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001baeec-2f5e-4b8f-afee-8ecca21e743f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:31.626856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001baeec-2f5e-4b8f-afee-8ecca21e743f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:31.626856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001baeec-2f5e-4b8f-afee-8ecca21e743f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:31.637110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 394/786] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:31.642550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68ca28-b26d-40ef-8bb2-d4d84c96e7f0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:31.645766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001baeec-2f5e-4b8f-afee-8ecca21e743f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:31.733687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:31.792559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68ca28-b26d-40ef-8bb2-d4d84c96e7f0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:31.838629 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001baeec-2f5e-4b8f-afee-8ecca21e743f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:31.851721 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b2467-3978-45ff-a6ff-c231d20b0f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:31.852627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b2467-3978-45ff-a6ff-c231d20b0f4b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:31.852912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b2467-3978-45ff-a6ff-c231d20b0f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:31.853184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b2467-3978-45ff-a6ff-c231d20b0f4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:31.863632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b2467-3978-45ff-a6ff-c231d20b0f4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:31.865197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b2467-3978-45ff-a6ff-c231d20b0f4b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-057b2467-3978-45ff-a6ff-c231d20b0f4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:31.883153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:31.884082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:31.884987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:31.884987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:31.884987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:31.884987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:31.884987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 393/787] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:31.890313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b2467-3978-45ff-a6ff-c231d20b0f4b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:31.898960 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e220858-a19f-431a-9e1e-6de8caacd39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:31.900010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220858-a19f-431a-9e1e-6de8caacd39c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:31.900512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220858-a19f-431a-9e1e-6de8caacd39c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:31.901009 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e220858-a19f-431a-9e1e-6de8caacd39c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:31.903935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:31.912449 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e220858-a19f-431a-9e1e-6de8caacd39c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0e220858-a19f-431a-9e1e-6de8caacd39c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:31.921218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:31.922174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:31.923096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:31.927119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 395/788] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:31.931303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e220858-a19f-431a-9e1e-6de8caacd39c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:32.053942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b2467-3978-45ff-a6ff-c231d20b0f4b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:32.074381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e220858-a19f-431a-9e1e-6de8caacd39c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:32.080391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:32.103288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d84fdb-36bd-43d3-814c-a3a7bfaa4bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:32.104108 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d84fdb-36bd-43d3-814c-a3a7bfaa4bcd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.104590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d84fdb-36bd-43d3-814c-a3a7bfaa4bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.104866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d84fdb-36bd-43d3-814c-a3a7bfaa4bcd None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.105778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d84fdb-36bd-43d3-814c-a3a7bfaa4bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:32.110785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d84fdb-36bd-43d3-814c-a3a7bfaa4bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:32.112757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d84fdb-36bd-43d3-814c-a3a7bfaa4bcd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:32.117327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ebf9b-a231-404a-9c61-73b837b76c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:32.119210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ebf9b-a231-404a-9c61-73b837b76c58 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.119642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ebf9b-a231-404a-9c61-73b837b76c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.120265 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ebf9b-a231-404a-9c61-73b837b76c58 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.124966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ebf9b-a231-404a-9c61-73b837b76c58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:32.125435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ebf9b-a231-404a-9c61-73b837b76c58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:32.126999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ebf9b-a231-404a-9c61-73b837b76c58 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:32.141682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 394/789] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:32.142117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 396/790] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:32.146509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d84fdb-36bd-43d3-814c-a3a7bfaa4bcd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:32.152500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ebf9b-a231-404a-9c61-73b837b76c58 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:32.223644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:32.270360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ebf9b-a231-404a-9c61-73b837b76c58 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:32.280296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d84fdb-36bd-43d3-814c-a3a7bfaa4bcd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:32.306587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49b81d-91fc-4e19-a246-83fd7c0bf873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:32.307380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b81d-91fc-4e19-a246-83fd7c0bf873 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.307663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b81d-91fc-4e19-a246-83fd7c0bf873 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.307934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b81d-91fc-4e19-a246-83fd7c0bf873 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.315751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49b81d-91fc-4e19-a246-83fd7c0bf873 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:32.317336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49b81d-91fc-4e19-a246-83fd7c0bf873 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:32.325930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080d236-dfd4-4397-93ab-bfac54b5cc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:32.327033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080d236-dfd4-4397-93ab-bfac54b5cc0d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.327519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080d236-dfd4-4397-93ab-bfac54b5cc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.327992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080d236-dfd4-4397-93ab-bfac54b5cc0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7a49b81d-91fc-4e19-a246-83fd7c0bf873 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:32.333097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:32.334041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:32.335035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:32.335035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:32.335035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:32.335035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:32.335035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:32.335035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:32.335035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:32.335035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:32.335035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:32.335035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:32.338945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 397/791] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:32.339896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff761f7-d0ce-4db8-afe7-44b5bcab2cee None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.340192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff761f7-d0ce-4db8-afe7-44b5bcab2cee None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.340476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff761f7-d0ce-4db8-afe7-44b5bcab2cee None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.343690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080d236-dfd4-4397-93ab-bfac54b5cc0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:32.345469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080d236-dfd4-4397-93ab-bfac54b5cc0d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:32.346929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 398/792] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:32.348401 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff761f7-d0ce-4db8-afe7-44b5bcab2cee None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-4080d236-dfd4-4397-93ab-bfac54b5cc0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:32.373351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:32.374322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:32.375377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:32.375377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:32.375377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:32.375377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:32.377393 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 395/793] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:32.378861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b6bb7-3679-4c19-9ec8-1e0ccaeaac00 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.379312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b6bb7-3679-4c19-9ec8-1e0ccaeaac00 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.379724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b6bb7-3679-4c19-9ec8-1e0ccaeaac00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.401226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5a9b6bb7-3679-4c19-9ec8-1e0ccaeaac00 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:32.402771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5a9b6bb7-3679-4c19-9ec8-1e0ccaeaac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:32.430290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:32.478469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 396/794] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:32.478469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b6bb7-3679-4c19-9ec8-1e0ccaeaac00 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:32.491112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff761f7-d0ce-4db8-afe7-44b5bcab2cee None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:32.523717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:32.546140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de10c48-4729-4c2f-91cc-03c7c6cef849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:32.547058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de10c48-4729-4c2f-91cc-03c7c6cef849 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.547508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de10c48-4729-4c2f-91cc-03c7c6cef849 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.547934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de10c48-4729-4c2f-91cc-03c7c6cef849 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.549032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de10c48-4729-4c2f-91cc-03c7c6cef849 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:32.549590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de10c48-4729-4c2f-91cc-03c7c6cef849 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:32.551212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de10c48-4729-4c2f-91cc-03c7c6cef849 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:32.578756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 399/795] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 26 20:51:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:32.580120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de10c48-4729-4c2f-91cc-03c7c6cef849 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:32.583160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b6bb7-3679-4c19-9ec8-1e0ccaeaac00 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:32.622527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247484af-e718-475b-b584-9d4d048498ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:32.623379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247484af-e718-475b-b584-9d4d048498ba None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.623765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247484af-e718-475b-b584-9d4d048498ba None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.624150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247484af-e718-475b-b584-9d4d048498ba None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.625192 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247484af-e718-475b-b584-9d4d048498ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:32.625406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247484af-e718-475b-b584-9d4d048498ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:32.626959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247484af-e718-475b-b584-9d4d048498ba None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:32.643521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 397/796] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:32.645125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247484af-e718-475b-b584-9d4d048498ba None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:32.672759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:32.705344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de10c48-4729-4c2f-91cc-03c7c6cef849 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:32.746783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572b8a5-ff99-44d9-89dc-50eb695228a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:32.748072 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572b8a5-ff99-44d9-89dc-50eb695228a1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.748543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572b8a5-ff99-44d9-89dc-50eb695228a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.748998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572b8a5-ff99-44d9-89dc-50eb695228a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.757268 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247484af-e718-475b-b584-9d4d048498ba None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:32.762190 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572b8a5-ff99-44d9-89dc-50eb695228a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c572b8a5-ff99-44d9-89dc-50eb695228a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:32.770698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:32.771656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:32.772548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:32.779856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 400/797] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:32.780959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572b8a5-ff99-44d9-89dc-50eb695228a1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:32.794087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc32917-3e75-48b0-83d8-76ef0cd6abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:32.795071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc32917-3e75-48b0-83d8-76ef0cd6abe9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.795455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc32917-3e75-48b0-83d8-76ef0cd6abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.795837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc32917-3e75-48b0-83d8-76ef0cd6abe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.809259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc32917-3e75-48b0-83d8-76ef0cd6abe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:32.810895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc32917-3e75-48b0-83d8-76ef0cd6abe9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:32.823992 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8bc32917-3e75-48b0-83d8-76ef0cd6abe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:32.830887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:32.831789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:32.837468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:32.837468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:32.837468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:32.837468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:32.837468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 398/798] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:32.837468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc32917-3e75-48b0-83d8-76ef0cd6abe9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:32.905577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572b8a5-ff99-44d9-89dc-50eb695228a1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:32.912542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:32.937888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc32917-3e75-48b0-83d8-76ef0cd6abe9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:32.945868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d782a-828a-4c34-a0f0-8fcdac0a0cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:32.946898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d782a-828a-4c34-a0f0-8fcdac0a0cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.947406 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d782a-828a-4c34-a0f0-8fcdac0a0cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.947830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d782a-828a-4c34-a0f0-8fcdac0a0cc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.956349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d782a-828a-4c34-a0f0-8fcdac0a0cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:32.957965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d782a-828a-4c34-a0f0-8fcdac0a0cc1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:32.972908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec50712-c3ba-4fff-b0ef-4f29e50ea187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:32.973696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec50712-c3ba-4fff-b0ef-4f29e50ea187 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:32.974090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec50712-c3ba-4fff-b0ef-4f29e50ea187 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-884d782a-828a-4c34-a0f0-8fcdac0a0cc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:32.974203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:32.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:32.976836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:32.976836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:32.976836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:32.976836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:32.976836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:32.976836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:32.976836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:32.976836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:32.976836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:51:32.976836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:32.977533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec50712-c3ba-4fff-b0ef-4f29e50ea187 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:32.977533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec50712-c3ba-4fff-b0ef-4f29e50ea187 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:32.977533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec50712-c3ba-4fff-b0ef-4f29e50ea187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:32.978595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec50712-c3ba-4fff-b0ef-4f29e50ea187 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:32.984251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 401/799] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 26 20:51:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:32.985454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d782a-828a-4c34-a0f0-8fcdac0a0cc1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:32.988922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 399/800] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:32.989067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec50712-c3ba-4fff-b0ef-4f29e50ea187 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:33.103099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:33.103652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec50712-c3ba-4fff-b0ef-4f29e50ea187 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:33.111027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d782a-828a-4c34-a0f0-8fcdac0a0cc1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:33.139050 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c42e3d-41fc-414d-9883-466803fccd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:33.139835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c42e3d-41fc-414d-9883-466803fccd29 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.140191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c42e3d-41fc-414d-9883-466803fccd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.140550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c42e3d-41fc-414d-9883-466803fccd29 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.141474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c42e3d-41fc-414d-9883-466803fccd29 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:33.141864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c42e3d-41fc-414d-9883-466803fccd29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:33.143407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c42e3d-41fc-414d-9883-466803fccd29 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:33.151103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e582c3d-f213-4048-8f0e-fb2b29e39979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:33.152035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e582c3d-f213-4048-8f0e-fb2b29e39979 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.152658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:33.153008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e582c3d-f213-4048-8f0e-fb2b29e39979 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.153465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e582c3d-f213-4048-8f0e-fb2b29e39979 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.155055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e582c3d-f213-4048-8f0e-fb2b29e39979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:33.155550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e582c3d-f213-4048-8f0e-fb2b29e39979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:33.157112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e582c3d-f213-4048-8f0e-fb2b29e39979 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:33.167657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 400/801] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 26 20:51:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:33.167657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c42e3d-41fc-414d-9883-466803fccd29 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:33.182122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 402/802] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:33.183455 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e582c3d-f213-4048-8f0e-fb2b29e39979 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:33.269406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c42e3d-41fc-414d-9883-466803fccd29 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:33.298850 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e582c3d-f213-4048-8f0e-fb2b29e39979 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:33.303916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d4684-fb00-4ae8-bd71-7b8c3b5d4ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:33.304746 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d4684-fb00-4ae8-bd71-7b8c3b5d4ceb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.305029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d4684-fb00-4ae8-bd71-7b8c3b5d4ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.305304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d4684-fb00-4ae8-bd71-7b8c3b5d4ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.315284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d4684-fb00-4ae8-bd71-7b8c3b5d4ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:33.316722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d4684-fb00-4ae8-bd71-7b8c3b5d4ceb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:33.334232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f816e5e-0fed-4891-babd-6edc84a58da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-3e5d4684-fb00-4ae8-bd71-7b8c3b5d4ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:33.334811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:33.336247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:33.337129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:33.337129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:33.337129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:33.337129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:33.337360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f816e5e-0fed-4891-babd-6edc84a58da4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.337360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f816e5e-0fed-4891-babd-6edc84a58da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.337360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f816e5e-0fed-4891-babd-6edc84a58da4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.337537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 401/803] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:33.338061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d4684-fb00-4ae8-bd71-7b8c3b5d4ceb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:33.365414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:33.395594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:33.411932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f816e5e-0fed-4891-babd-6edc84a58da4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:33.413456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f816e5e-0fed-4891-babd-6edc84a58da4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:33.453207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d4684-fb00-4ae8-bd71-7b8c3b5d4ceb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:33.490562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b14cc5-cb32-472d-b8ee-cc3c22e167ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:33.491362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b14cc5-cb32-472d-b8ee-cc3c22e167ec None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.491684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b14cc5-cb32-472d-b8ee-cc3c22e167ec None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.491965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b14cc5-cb32-472d-b8ee-cc3c22e167ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.499007 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b14cc5-cb32-472d-b8ee-cc3c22e167ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:33.500455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b14cc5-cb32-472d-b8ee-cc3c22e167ec None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-41b14cc5-cb32-472d-b8ee-cc3c22e167ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:33.515073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:33.515957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:33.516850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:33.516850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:33.516850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:33.516850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:33.516850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:33.516850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:33.516850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:33.516850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:33.516850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 402/804] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:33.517642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abecc5a-0611-43c5-a679-2e323512b671 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.517733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abecc5a-0611-43c5-a679-2e323512b671 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.518003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abecc5a-0611-43c5-a679-2e323512b671 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.522478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 403/805] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:33.522478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abecc5a-0611-43c5-a679-2e323512b671 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:33.530031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 403/806] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:51:33.531047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f816e5e-0fed-4891-babd-6edc84a58da4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:33.593806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:33.638323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abecc5a-0611-43c5-a679-2e323512b671 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:33.646693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f816e5e-0fed-4891-babd-6edc84a58da4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:33.647572 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:33.674945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c62f8-4a2c-48ba-aaf0-88f582c7b52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:33.675633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c62f8-4a2c-48ba-aaf0-88f582c7b52f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.675922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c62f8-4a2c-48ba-aaf0-88f582c7b52f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.676199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c62f8-4a2c-48ba-aaf0-88f582c7b52f None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.676916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c62f8-4a2c-48ba-aaf0-88f582c7b52f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:33.677247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c62f8-4a2c-48ba-aaf0-88f582c7b52f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:33.678658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c62f8-4a2c-48ba-aaf0-88f582c7b52f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:33.684701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e5386-4b79-43d2-989a-74ed30d2f9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:33.685368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e5386-4b79-43d2-989a-74ed30d2f9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.686028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e5386-4b79-43d2-989a-74ed30d2f9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.686028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e5386-4b79-43d2-989a-74ed30d2f9b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.686746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e5386-4b79-43d2-989a-74ed30d2f9b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:33.687066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e5386-4b79-43d2-989a-74ed30d2f9b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:33.688511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e5386-4b79-43d2-989a-74ed30d2f9b4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:33.691129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 404/807] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:33.692425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453d9d4-fca3-428c-85d6-c993f9f9a7ae None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.692728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453d9d4-fca3-428c-85d6-c993f9f9a7ae None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.693006 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453d9d4-fca3-428c-85d6-c993f9f9a7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.705297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 404/808] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:33.706175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e5386-4b79-43d2-989a-74ed30d2f9b4 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:33.713586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e453d9d4-fca3-428c-85d6-c993f9f9a7ae None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:33.714814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e453d9d4-fca3-428c-85d6-c993f9f9a7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:33.782580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 405/809] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:33.791068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453d9d4-fca3-428c-85d6-c993f9f9a7ae None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:33.825634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e5386-4b79-43d2-989a-74ed30d2f9b4 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:33.842253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:33.860271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed52480-1391-4f1a-b5aa-0872be94fbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:33.861053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed52480-1391-4f1a-b5aa-0872be94fbfd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.861330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed52480-1391-4f1a-b5aa-0872be94fbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.861603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed52480-1391-4f1a-b5aa-0872be94fbfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.871939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed52480-1391-4f1a-b5aa-0872be94fbfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:33.873391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed52480-1391-4f1a-b5aa-0872be94fbfd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:33.887608 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9ed52480-1391-4f1a-b5aa-0872be94fbfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:33.890421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:33.891446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:33.892372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:33.892372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:33.892372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:33.892372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:33.892372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 405/810] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:33.893006 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453d9d4-fca3-428c-85d6-c993f9f9a7ae None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:33.893099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed52480-1391-4f1a-b5aa-0872be94fbfd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:33.926254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78ea86-5019-4519-868a-165856da458c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:33.927086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78ea86-5019-4519-868a-165856da458c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.927367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78ea86-5019-4519-868a-165856da458c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.927641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78ea86-5019-4519-868a-165856da458c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.935362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78ea86-5019-4519-868a-165856da458c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:33.936809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78ea86-5019-4519-868a-165856da458c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-af78ea86-5019-4519-868a-165856da458c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:33.951499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:33.952427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:33.953521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 406/811] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 26 20:51:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:33.954272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aae51-cd0d-4ada-80e2-59f6796f5968 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:33.954272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aae51-cd0d-4ada-80e2-59f6796f5968 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:33.954535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6aae51-cd0d-4ada-80e2-59f6796f5968 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:33.955879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 407/812] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:33.957977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6aae51-cd0d-4ada-80e2-59f6796f5968 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:33.996488 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed52480-1391-4f1a-b5aa-0872be94fbfd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.029457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0cbf4-8b6f-4e02-85c6-16230679a747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.030338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0cbf4-8b6f-4e02-85c6-16230679a747 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.030653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0cbf4-8b6f-4e02-85c6-16230679a747 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.030939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0cbf4-8b6f-4e02-85c6-16230679a747 None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/1285406084604bd19287de9e1beeeaa8/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.032911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0cbf4-8b6f-4e02-85c6-16230679a747 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=1285406084604bd19287de9e1beeeaa8, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.034486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0cbf4-8b6f-4e02-85c6-16230679a747 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:34.048077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 406/813] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:51:33 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/1285406084604bd19287de9e1beeeaa8/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:51:34.049022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0cbf4-8b6f-4e02-85c6-16230679a747 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:34.069062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6aae51-cd0d-4ada-80e2-59f6796f5968 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.083406 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:34.111180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2ce4d-11ee-40f4-bf79-f9ef03ca0230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.112013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2ce4d-11ee-40f4-bf79-f9ef03ca0230 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.112304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2ce4d-11ee-40f4-bf79-f9ef03ca0230 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.112581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2ce4d-11ee-40f4-bf79-f9ef03ca0230 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.113245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2ce4d-11ee-40f4-bf79-f9ef03ca0230 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.114711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2ce4d-11ee-40f4-bf79-f9ef03ca0230 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0db2ce4d-11ee-40f4-bf79-f9ef03ca0230 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:34.127824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:34.128895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:34.135078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 408/814] 10.222.0.57 () {56 vars in 1129 bytes} [Thu May 26 20:51:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:34.136282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2ce4d-11ee-40f4-bf79-f9ef03ca0230 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:34.162798 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:34.177769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0cbf4-8b6f-4e02-85c6-16230679a747 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.217685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9b86d-064b-413f-b419-45092cdf3ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.218360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9b86d-064b-413f-b419-45092cdf3ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.218669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9b86d-064b-413f-b419-45092cdf3ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.218949 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9b86d-064b-413f-b419-45092cdf3ef5 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.219676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9b86d-064b-413f-b419-45092cdf3ef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:34.219981 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9b86d-064b-413f-b419-45092cdf3ef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.222110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded9b86d-064b-413f-b419-45092cdf3ef5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:34.240106 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 407/815] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:34.241395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a076962-449e-449c-b1c5-6ef064d35cb8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.242245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a076962-449e-449c-b1c5-6ef064d35cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.242548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a076962-449e-449c-b1c5-6ef064d35cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.252051 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2ce4d-11ee-40f4-bf79-f9ef03ca0230 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.261767 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3a076962-449e-449c-b1c5-6ef064d35cb8 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:34.263029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3a076962-449e-449c-b1c5-6ef064d35cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:34.287275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f1290-5909-4dcf-b072-5a59ff53c970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.288001 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f1290-5909-4dcf-b072-5a59ff53c970 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.288795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f1290-5909-4dcf-b072-5a59ff53c970 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.289119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f1290-5909-4dcf-b072-5a59ff53c970 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.289970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f1290-5909-4dcf-b072-5a59ff53c970 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:34.290264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f1290-5909-4dcf-b072-5a59ff53c970 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.291733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f1290-5909-4dcf-b072-5a59ff53c970 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:34.310211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 409/816] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 26 20:51:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:34.315135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f1290-5909-4dcf-b072-5a59ff53c970 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:34.335951 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 408/817] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:34.337854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a076962-449e-449c-b1c5-6ef064d35cb8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:34.367175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:34.405826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:34.437109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f1290-5909-4dcf-b072-5a59ff53c970 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.477661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a076962-449e-449c-b1c5-6ef064d35cb8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.481555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5a489-c73e-4979-9a67-8c451db6ca8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.482225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5a489-c73e-4979-9a67-8c451db6ca8b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.482546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5a489-c73e-4979-9a67-8c451db6ca8b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.482826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5a489-c73e-4979-9a67-8c451db6ca8b None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.483532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5a489-c73e-4979-9a67-8c451db6ca8b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:34.483839 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5a489-c73e-4979-9a67-8c451db6ca8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.485334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5a489-c73e-4979-9a67-8c451db6ca8b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:34.495305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 410/818] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 26 20:51:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:34.495305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5a489-c73e-4979-9a67-8c451db6ca8b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:34.519246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6924e7-ef52-4d4c-8416-77e37a1d6174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.520152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6924e7-ef52-4d4c-8416-77e37a1d6174 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.520511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6924e7-ef52-4d4c-8416-77e37a1d6174 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.520863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6924e7-ef52-4d4c-8416-77e37a1d6174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.528169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6924e7-ef52-4d4c-8416-77e37a1d6174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.529628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6924e7-ef52-4d4c-8416-77e37a1d6174 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0c6924e7-ef52-4d4c-8416-77e37a1d6174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:34.544037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:34.545025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:34.545929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:34.545929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:34.545929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:34.545929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:34.545929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:34.545929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:34.545929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:34.545929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:34.549978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 409/819] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:34.550836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6924e7-ef52-4d4c-8416-77e37a1d6174 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:34.595060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5a489-c73e-4979-9a67-8c451db6ca8b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.631918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:34.635068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b3cb9-7e51-47b2-a50b-3f7e30b8f79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.636188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b3cb9-7e51-47b2-a50b-3f7e30b8f79d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.636478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b3cb9-7e51-47b2-a50b-3f7e30b8f79d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.636849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b3cb9-7e51-47b2-a50b-3f7e30b8f79d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.648276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b3cb9-7e51-47b2-a50b-3f7e30b8f79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.649774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b3cb9-7e51-47b2-a50b-3f7e30b8f79d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:34.652324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:34.656565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6924e7-ef52-4d4c-8416-77e37a1d6174 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7f5b3cb9-7e51-47b2-a50b-3f7e30b8f79d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:34.669037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:34.670126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:34.675742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:34.675742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:34.675742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:34.675742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:34.675742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 411/820] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:34.675742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b3cb9-7e51-47b2-a50b-3f7e30b8f79d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:34.703173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f8cd0-f255-4bb8-8c25-ca21a66456f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.703889 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f8cd0-f255-4bb8-8c25-ca21a66456f5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.704175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f8cd0-f255-4bb8-8c25-ca21a66456f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.704453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f8cd0-f255-4bb8-8c25-ca21a66456f5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.705108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f8cd0-f255-4bb8-8c25-ca21a66456f5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.706742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f8cd0-f255-4bb8-8c25-ca21a66456f5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f47f8cd0-f255-4bb8-8c25-ca21a66456f5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:34.718657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:34.719668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:34.720519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 410/821] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:34 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:34.721319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f8cd0-f255-4bb8-8c25-ca21a66456f5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:34.784812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b3cb9-7e51-47b2-a50b-3f7e30b8f79d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.826960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8165ec-d573-41d7-a15c-a346e9e0e5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.827667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8165ec-d573-41d7-a15c-a346e9e0e5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.827950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8165ec-d573-41d7-a15c-a346e9e0e5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.828227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8165ec-d573-41d7-a15c-a346e9e0e5a4 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.828932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8165ec-d573-41d7-a15c-a346e9e0e5a4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:34.829239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8165ec-d573-41d7-a15c-a346e9e0e5a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.830633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8165ec-d573-41d7-a15c-a346e9e0e5a4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:34.834683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f8cd0-f255-4bb8-8c25-ca21a66456f5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.836050 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 412/822] 10.222.0.57 () {56 vars in 1124 bytes} [Thu May 26 20:51:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:34.839443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8165ec-d573-41d7-a15c-a346e9e0e5a4 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:34.873092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bf49c-2b91-4c14-a7f1-cf79e2eda59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.873755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf49c-2b91-4c14-a7f1-cf79e2eda59a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.874038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf49c-2b91-4c14-a7f1-cf79e2eda59a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.874314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bf49c-2b91-4c14-a7f1-cf79e2eda59a None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.876005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bf49c-2b91-4c14-a7f1-cf79e2eda59a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:34.876343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bf49c-2b91-4c14-a7f1-cf79e2eda59a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.877794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50bf49c-2b91-4c14-a7f1-cf79e2eda59a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:34.897810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 411/823] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:34.897810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bf49c-2b91-4c14-a7f1-cf79e2eda59a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:34.900730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:34.901319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:34.945884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8165ec-d573-41d7-a15c-a346e9e0e5a4 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:34.985691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614d815-46e0-4ddd-b8cd-279eb02457bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:34.986722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614d815-46e0-4ddd-b8cd-279eb02457bd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:34.987198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614d815-46e0-4ddd-b8cd-279eb02457bd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:34.987661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614d815-46e0-4ddd-b8cd-279eb02457bd None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:34.988566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614d815-46e0-4ddd-b8cd-279eb02457bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:34.989059 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614d815-46e0-4ddd-b8cd-279eb02457bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:34.990655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614d815-46e0-4ddd-b8cd-279eb02457bd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:35.004917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 413/824] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:35.005783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614d815-46e0-4ddd-b8cd-279eb02457bd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:35.014188 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bf49c-2b91-4c14-a7f1-cf79e2eda59a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:35.050091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90559af9-7e8c-4f75-a963-b89a36054dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:35.051154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90559af9-7e8c-4f75-a963-b89a36054dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:35.051587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90559af9-7e8c-4f75-a963-b89a36054dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:35.052010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90559af9-7e8c-4f75-a963-b89a36054dc1 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:35.052915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90559af9-7e8c-4f75-a963-b89a36054dc1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:35.053229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90559af9-7e8c-4f75-a963-b89a36054dc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:35.055244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90559af9-7e8c-4f75-a963-b89a36054dc1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:35.064240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 412/825] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:35.064240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90559af9-7e8c-4f75-a963-b89a36054dc1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:35.117307 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614d815-46e0-4ddd-b8cd-279eb02457bd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:35.133570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:35.138333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:35.159951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f90f6f-409f-42d5-8eca-69ed8bc21f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:35.160632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f90f6f-409f-42d5-8eca-69ed8bc21f4c None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:35.160920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f90f6f-409f-42d5-8eca-69ed8bc21f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:35.161199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f90f6f-409f-42d5-8eca-69ed8bc21f4c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:35.161990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f90f6f-409f-42d5-8eca-69ed8bc21f4c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:35.163949 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f90f6f-409f-42d5-8eca-69ed8bc21f4c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:35.175853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90559af9-7e8c-4f75-a963-b89a36054dc1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:35.187648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 414/826] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 26 20:51:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:51:35.188551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f90f6f-409f-42d5-8eca-69ed8bc21f4c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:35.236616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f9c06-2a30-456d-984b-21ab7918e700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:35.237943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f9c06-2a30-456d-984b-21ab7918e700 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:35.238414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f9c06-2a30-456d-984b-21ab7918e700 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:35.239039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f9c06-2a30-456d-984b-21ab7918e700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:35.240196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f9c06-2a30-456d-984b-21ab7918e700 None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=490d773bf21446789b77c5793a518a5a) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:35.240701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f9c06-2a30-456d-984b-21ab7918e700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:35.243112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f9c06-2a30-456d-984b-21ab7918e700 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:35.259035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 413/827] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:35 2022] GET /identity/v3/role_assignments?user.id=490d773bf21446789b77c5793a518a5a&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:35.260287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f9c06-2a30-456d-984b-21ab7918e700 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:35.404073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f90f6f-409f-42d5-8eca-69ed8bc21f4c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:35.453018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb7d44-5b68-4923-85a7-4900516d8afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:35.453829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb7d44-5b68-4923-85a7-4900516d8afa None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:35.454005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f9c06-2a30-456d-984b-21ab7918e700 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:35.454106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb7d44-5b68-4923-85a7-4900516d8afa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:35.454377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb7d44-5b68-4923-85a7-4900516d8afa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:35.463265 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb7d44-5b68-4923-85a7-4900516d8afa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:35.464736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb7d44-5b68-4923-85a7-4900516d8afa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-07bb7d44-5b68-4923-85a7-4900516d8afa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:35.481261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:35.482208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:35.483176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:35.483176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:35.483176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:35.483176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:35.483176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:35.483176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:35.483176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:35.483176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:35.484153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 415/828] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:35.490412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb7d44-5b68-4923-85a7-4900516d8afa None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:35.493717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:35.512200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b414ac-3d2e-40ed-a38d-1c1423fc7f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:35.512867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b414ac-3d2e-40ed-a38d-1c1423fc7f9c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:35.513155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b414ac-3d2e-40ed-a38d-1c1423fc7f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:35.513431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b414ac-3d2e-40ed-a38d-1c1423fc7f9c None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:35.514145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b414ac-3d2e-40ed-a38d-1c1423fc7f9c None admin] RBAC: Adding query filter params (type=network) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:35.514467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b414ac-3d2e-40ed-a38d-1c1423fc7f9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:35.515966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b414ac-3d2e-40ed-a38d-1c1423fc7f9c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:35.523808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 414/829] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:35.531045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b414ac-3d2e-40ed-a38d-1c1423fc7f9c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:35.539002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:35.620369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb7d44-5b68-4923-85a7-4900516d8afa None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:35.658160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5b16e-1dd5-46fe-bdbf-25a47499a23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:35.658862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5b16e-1dd5-46fe-bdbf-25a47499a23a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:35.659149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5b16e-1dd5-46fe-bdbf-25a47499a23a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:35.659428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5b16e-1dd5-46fe-bdbf-25a47499a23a None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:35.660218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5b16e-1dd5-46fe-bdbf-25a47499a23a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:35.660530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5b16e-1dd5-46fe-bdbf-25a47499a23a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:35.661962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5b16e-1dd5-46fe-bdbf-25a47499a23a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:35.668304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b414ac-3d2e-40ed-a38d-1c1423fc7f9c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:35.674886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 416/830] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:35.683593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5b16e-1dd5-46fe-bdbf-25a47499a23a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:35.713481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301b8f0-2a85-475a-9044-54adb0722318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:35.714149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301b8f0-2a85-475a-9044-54adb0722318 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:35.714542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301b8f0-2a85-475a-9044-54adb0722318 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:35.714742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301b8f0-2a85-475a-9044-54adb0722318 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:35.715494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301b8f0-2a85-475a-9044-54adb0722318 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:35.716962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301b8f0-2a85-475a-9044-54adb0722318 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:35.750610 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 415/831] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 26 20:51:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:51:35.823863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:35.836739 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5b16e-1dd5-46fe-bdbf-25a47499a23a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:35.896997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bad272-73c1-4b9f-ad7d-ae99c6667003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:35.898493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bad272-73c1-4b9f-ad7d-ae99c6667003 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:35.898838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bad272-73c1-4b9f-ad7d-ae99c6667003 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:35.899126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bad272-73c1-4b9f-ad7d-ae99c6667003 None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/e67f497e8a8b46e1b665999ba8fda38d/roles/4e24596391d3466eacb50a9e923e07ab` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:35.918781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bad272-73c1-4b9f-ad7d-ae99c6667003 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=e67f497e8a8b46e1b665999ba8fda38d, role_id=4e24596391d3466eacb50a9e923e07ab)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:35.920333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bad272-73c1-4b9f-ad7d-ae99c6667003 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:35.947837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 417/832] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:51:35 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/e67f497e8a8b46e1b665999ba8fda38d/roles/4e24596391d3466eacb50a9e923e07ab => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:51:36.349924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71114662-9a96-4205-9bde-b626d080be59 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:36.350434 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71114662-9a96-4205-9bde-b626d080be59 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:36.350541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71114662-9a96-4205-9bde-b626d080be59 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:36.352116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 416/833] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:36.358233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45856-152a-45bb-b80a-d6dac5b3e053 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:36.358552 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45856-152a-45bb-b80a-d6dac5b3e053 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:36.358838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45856-152a-45bb-b80a-d6dac5b3e053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:36.380027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-15f45856-152a-45bb-b80a-d6dac5b3e053 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:36.381239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-15f45856-152a-45bb-b80a-d6dac5b3e053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:36.452710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 418/834] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:36.475408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71114662-9a96-4205-9bde-b626d080be59 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:36.525864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:36.587609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71114662-9a96-4205-9bde-b626d080be59 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:36.627544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77307391-88ed-4360-9fed-29f475228d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:36.628722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77307391-88ed-4360-9fed-29f475228d53 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:36.629215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77307391-88ed-4360-9fed-29f475228d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:36.629699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77307391-88ed-4360-9fed-29f475228d53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:36.641628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77307391-88ed-4360-9fed-29f475228d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-77307391-88ed-4360-9fed-29f475228d53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:36.649720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:36.650930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:51:36.651858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:36.654563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 417/835] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:36.664624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f45856-152a-45bb-b80a-d6dac5b3e053 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:36.740275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:36.773891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f45856-152a-45bb-b80a-d6dac5b3e053 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:36.817563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225757a-b01f-4bc7-a276-85c9a38fcedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:36.818244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225757a-b01f-4bc7-a276-85c9a38fcedb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:36.818570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225757a-b01f-4bc7-a276-85c9a38fcedb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:36.818858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225757a-b01f-4bc7-a276-85c9a38fcedb None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:36.819641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225757a-b01f-4bc7-a276-85c9a38fcedb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:36.819952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225757a-b01f-4bc7-a276-85c9a38fcedb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:36.821325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225757a-b01f-4bc7-a276-85c9a38fcedb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:36.830980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 419/836] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:36.836890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77307391-88ed-4360-9fed-29f475228d53 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:36.946409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77307391-88ed-4360-9fed-29f475228d53 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:36.953831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:36.982670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693641a-446f-435e-a280-989adf800a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:36.983466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693641a-446f-435e-a280-989adf800a32 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:36.983753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693641a-446f-435e-a280-989adf800a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:36.984037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693641a-446f-435e-a280-989adf800a32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:36.994306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693641a-446f-435e-a280-989adf800a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:36.995887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693641a-446f-435e-a280-989adf800a32 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6693641a-446f-435e-a280-989adf800a32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:37.017049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:37.017999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:37.018941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:37.018941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:37.018941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:37.018941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:37.018941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 418/837] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:37.024511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225757a-b01f-4bc7-a276-85c9a38fcedb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:37.125250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225757a-b01f-4bc7-a276-85c9a38fcedb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:37.159183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706ced7-562f-4741-af6e-010372fd7a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:37.159840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706ced7-562f-4741-af6e-010372fd7a80 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:37.160119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706ced7-562f-4741-af6e-010372fd7a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:37.160394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706ced7-562f-4741-af6e-010372fd7a80 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:37.161107 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706ced7-562f-4741-af6e-010372fd7a80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:37.161411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706ced7-562f-4741-af6e-010372fd7a80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:37.162817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706ced7-562f-4741-af6e-010372fd7a80 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:37.164814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:37.175625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 420/838] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:37.181429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693641a-446f-435e-a280-989adf800a32 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:37.285467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693641a-446f-435e-a280-989adf800a32 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:37.322957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124f275-1d18-4cc7-b4fc-72d56c02e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:37.323755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124f275-1d18-4cc7-b4fc-72d56c02e841 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:37.324035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124f275-1d18-4cc7-b4fc-72d56c02e841 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:37.324310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124f275-1d18-4cc7-b4fc-72d56c02e841 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:37.333037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124f275-1d18-4cc7-b4fc-72d56c02e841 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:37.334510 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124f275-1d18-4cc7-b4fc-72d56c02e841 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0124f275-1d18-4cc7-b4fc-72d56c02e841 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:37.349238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:37.351869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:37.354177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 419/839] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:51:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:37.362464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706ced7-562f-4741-af6e-010372fd7a80 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:37.457909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706ced7-562f-4741-af6e-010372fd7a80 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:37.494129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a2b2f-a778-4f1b-a0b7-31ceeba3ad1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:37.494857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a2b2f-a778-4f1b-a0b7-31ceeba3ad1a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:37.495149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a2b2f-a778-4f1b-a0b7-31ceeba3ad1a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:37.495429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a2b2f-a778-4f1b-a0b7-31ceeba3ad1a None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:37.496279 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a2b2f-a778-4f1b-a0b7-31ceeba3ad1a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:37.496591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a2b2f-a778-4f1b-a0b7-31ceeba3ad1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:37.498061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a2b2f-a778-4f1b-a0b7-31ceeba3ad1a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:37.519503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 421/840] 10.222.0.57 () {56 vars in 1135 bytes} [Thu May 26 20:51:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:37.525738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124f275-1d18-4cc7-b4fc-72d56c02e841 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:37.549126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:37.636996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124f275-1d18-4cc7-b4fc-72d56c02e841 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:37.671989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde65325-3fea-48f4-8b48-a91003e799f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:37.672885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde65325-3fea-48f4-8b48-a91003e799f6 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:37.673322 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde65325-3fea-48f4-8b48-a91003e799f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:37.673749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde65325-3fea-48f4-8b48-a91003e799f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:37.688256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e75e8-7f22-49c2-a2d5-8a32c10169af None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:37.688549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e75e8-7f22-49c2-a2d5-8a32c10169af None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:37.688833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e75e8-7f22-49c2-a2d5-8a32c10169af None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:37.690430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 422/841] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:37.695471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef57db-bb38-4f66-bcdf-b0fa987fc677 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:37.695964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef57db-bb38-4f66-bcdf-b0fa987fc677 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:37.696313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef57db-bb38-4f66-bcdf-b0fa987fc677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:37.716918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2bef57db-bb38-4f66-bcdf-b0fa987fc677 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:37.718666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2bef57db-bb38-4f66-bcdf-b0fa987fc677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:37.748904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde65325-3fea-48f4-8b48-a91003e799f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:37.750681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde65325-3fea-48f4-8b48-a91003e799f6 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:37.770674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:37.802966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:37.808634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 423/842] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:37.834421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef57db-bb38-4f66-bcdf-b0fa987fc677 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:37.884031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 420/843] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:37 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:51:37.889761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde65325-3fea-48f4-8b48-a91003e799f6 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:37.943113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef57db-bb38-4f66-bcdf-b0fa987fc677 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:37.979967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eececd1-f485-49e7-8b58-6754fb852cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:37.980946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eececd1-f485-49e7-8b58-6754fb852cbf None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:37.981335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eececd1-f485-49e7-8b58-6754fb852cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:37.981715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eececd1-f485-49e7-8b58-6754fb852cbf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:37.982511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eececd1-f485-49e7-8b58-6754fb852cbf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:37.983993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eececd1-f485-49e7-8b58-6754fb852cbf None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-0eececd1-f485-49e7-8b58-6754fb852cbf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:37.997263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:37.998547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:51:37.999576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:38.006384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 424/844] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:51:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:38.007792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eececd1-f485-49e7-8b58-6754fb852cbf None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:38.015421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde65325-3fea-48f4-8b48-a91003e799f6 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:38.034591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:38.052792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eced5f-3044-47ab-8df3-3b73d9e14477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:38.053628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eced5f-3044-47ab-8df3-3b73d9e14477 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.054400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eced5f-3044-47ab-8df3-3b73d9e14477 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.054700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eced5f-3044-47ab-8df3-3b73d9e14477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.059137 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:38.065542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eced5f-3044-47ab-8df3-3b73d9e14477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:38.067039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eced5f-3044-47ab-8df3-3b73d9e14477 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-43eced5f-3044-47ab-8df3-3b73d9e14477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:38.084895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:38.085877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:38.086823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:38.086823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:38.086823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:38.086823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:38.086823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 421/845] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:38.087452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe2476-63ad-44b3-915e-cd6c2b3c09d3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.087653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe2476-63ad-44b3-915e-cd6c2b3c09d3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.087929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe2476-63ad-44b3-915e-cd6c2b3c09d3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.089428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 422/846] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:38.091396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe2476-63ad-44b3-915e-cd6c2b3c09d3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:38.122332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eececd1-f485-49e7-8b58-6754fb852cbf None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:38.162316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce3d81-2613-43e5-9bb3-644e0de46304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:38.163026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce3d81-2613-43e5-9bb3-644e0de46304 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.163841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce3d81-2613-43e5-9bb3-644e0de46304 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.164133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce3d81-2613-43e5-9bb3-644e0de46304 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.164838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce3d81-2613-43e5-9bb3-644e0de46304 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:38.165150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce3d81-2613-43e5-9bb3-644e0de46304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:38.166602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce3d81-2613-43e5-9bb3-644e0de46304 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:38.172684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 425/847] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:51:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:38.173922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d6f1-e8b0-4dc6-9568-c685edfb9277 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.174219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d6f1-e8b0-4dc6-9568-c685edfb9277 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.174521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d6f1-e8b0-4dc6-9568-c685edfb9277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.192959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-57a4d6f1-e8b0-4dc6-9568-c685edfb9277 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:38.194248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-57a4d6f1-e8b0-4dc6-9568-c685edfb9277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:38.208851 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe2476-63ad-44b3-915e-cd6c2b3c09d3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:38.246047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7a126-1463-45a5-9fc6-6c1ad5659cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:38.246731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7a126-1463-45a5-9fc6-6c1ad5659cdb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.247053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7a126-1463-45a5-9fc6-6c1ad5659cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.247326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc7a126-1463-45a5-9fc6-6c1ad5659cdb None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.248040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7a126-1463-45a5-9fc6-6c1ad5659cdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:38.248346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7a126-1463-45a5-9fc6-6c1ad5659cdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:38.249721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc7a126-1463-45a5-9fc6-6c1ad5659cdb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:38.261798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 423/848] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:38.262861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7a126-1463-45a5-9fc6-6c1ad5659cdb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:38.270744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 426/849] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:38.271666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4d6f1-e8b0-4dc6-9568-c685edfb9277 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:38.303441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:38.324321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:38.390089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc7a126-1463-45a5-9fc6-6c1ad5659cdb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:38.401546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4d6f1-e8b0-4dc6-9568-c685edfb9277 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:38.422782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b929077-290c-4563-b166-77ea8bdeded3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:38.423443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b929077-290c-4563-b166-77ea8bdeded3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.423728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b929077-290c-4563-b166-77ea8bdeded3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.424006 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b929077-290c-4563-b166-77ea8bdeded3 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.424704 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b929077-290c-4563-b166-77ea8bdeded3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:38.425015 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b929077-290c-4563-b166-77ea8bdeded3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:38.426394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b929077-290c-4563-b166-77ea8bdeded3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:38.430895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 424/850] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:51:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:38.431658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b929077-290c-4563-b166-77ea8bdeded3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:38.435164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fb7fe-94cf-48f0-bd6b-12c078c30e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:38.435989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fb7fe-94cf-48f0-bd6b-12c078c30e2c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.436276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fb7fe-94cf-48f0-bd6b-12c078c30e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.436555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fb7fe-94cf-48f0-bd6b-12c078c30e2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.444590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fb7fe-94cf-48f0-bd6b-12c078c30e2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:38.446109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fb7fe-94cf-48f0-bd6b-12c078c30e2c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5d8fb7fe-94cf-48f0-bd6b-12c078c30e2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:38.461588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:38.462627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:38.463533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:38.463533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:38.463533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:38.463533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:38.463533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:38.463533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:38.463533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:38.463533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:38.466618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 427/851] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:38.468485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070f871-f32d-49d2-af35-3e01741c6ec2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.468809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070f871-f32d-49d2-af35-3e01741c6ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.469095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070f871-f32d-49d2-af35-3e01741c6ec2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.470658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 428/852] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:38.475811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070f871-f32d-49d2-af35-3e01741c6ec2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:38.541728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:38.549051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b929077-290c-4563-b166-77ea8bdeded3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:38.584934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad7566-5975-402a-94d5-1f51e360d75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:38.586194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad7566-5975-402a-94d5-1f51e360d75c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.586495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad7566-5975-402a-94d5-1f51e360d75c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.586774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad7566-5975-402a-94d5-1f51e360d75c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.587435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad7566-5975-402a-94d5-1f51e360d75c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:38.588846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad7566-5975-402a-94d5-1f51e360d75c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:38.601528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-cfad7566-5975-402a-94d5-1f51e360d75c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:38.602112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:38.603275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:38.604123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 425/853] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 26 20:51:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:38.604975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83605d6-18e9-4cc4-8118-8f0da1edd581 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.604975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83605d6-18e9-4cc4-8118-8f0da1edd581 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.605200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83605d6-18e9-4cc4-8118-8f0da1edd581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.615074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070f871-f32d-49d2-af35-3e01741c6ec2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:38.623716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f83605d6-18e9-4cc4-8118-8f0da1edd581 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:38.624934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f83605d6-18e9-4cc4-8118-8f0da1edd581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:38.659169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6ae63-c7e3-4d27-aa19-e8b81a638d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:38.659838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6ae63-c7e3-4d27-aa19-e8b81a638d4b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.660326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6ae63-c7e3-4d27-aa19-e8b81a638d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.660603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6ae63-c7e3-4d27-aa19-e8b81a638d4b None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.661391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6ae63-c7e3-4d27-aa19-e8b81a638d4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:38.661701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6ae63-c7e3-4d27-aa19-e8b81a638d4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:38.663920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6ae63-c7e3-4d27-aa19-e8b81a638d4b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:38.689713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 429/854] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:38.689713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6ae63-c7e3-4d27-aa19-e8b81a638d4b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:38.718605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 426/855] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:38.718605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83605d6-18e9-4cc4-8118-8f0da1edd581 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:38.793520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6ae63-c7e3-4d27-aa19-e8b81a638d4b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:38.820606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:38.835111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83605d6-18e9-4cc4-8118-8f0da1edd581 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:38.836491 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5acc-7b0f-47ff-bd5e-3558bedb87df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:38.837161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5acc-7b0f-47ff-bd5e-3558bedb87df None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.837454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5acc-7b0f-47ff-bd5e-3558bedb87df None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.837735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b5acc-7b0f-47ff-bd5e-3558bedb87df None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.838451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b5acc-7b0f-47ff-bd5e-3558bedb87df None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:38.838784 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b5acc-7b0f-47ff-bd5e-3558bedb87df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:38.840155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b5acc-7b0f-47ff-bd5e-3558bedb87df None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:38.844623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 430/856] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:38.845391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5acc-7b0f-47ff-bd5e-3558bedb87df None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:38.870552 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:38.878089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937802d5-51c9-4d4e-b180-d7841e2633cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:38.879022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937802d5-51c9-4d4e-b180-d7841e2633cd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:38.879530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937802d5-51c9-4d4e-b180-d7841e2633cd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:38.880002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937802d5-51c9-4d4e-b180-d7841e2633cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:38.881045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937802d5-51c9-4d4e-b180-d7841e2633cd None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=1285406084604bd19287de9e1beeeaa8) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:38.881430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937802d5-51c9-4d4e-b180-d7841e2633cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:38.883500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937802d5-51c9-4d4e-b180-d7841e2633cd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:38.898475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 427/857] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:38 2022] GET /identity/v3/role_assignments?user.id=1285406084604bd19287de9e1beeeaa8&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:38.898475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937802d5-51c9-4d4e-b180-d7841e2633cd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:38.978856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b5acc-7b0f-47ff-bd5e-3558bedb87df None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:39.017895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07a7d9-652f-4507-ad43-2af8c0cfd785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:39.018737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07a7d9-652f-4507-ad43-2af8c0cfd785 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:39.019024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07a7d9-652f-4507-ad43-2af8c0cfd785 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:39.019303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07a7d9-652f-4507-ad43-2af8c0cfd785 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:39.024196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937802d5-51c9-4d4e-b180-d7841e2633cd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:39.031649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07a7d9-652f-4507-ad43-2af8c0cfd785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-7d07a7d9-652f-4507-ad43-2af8c0cfd785 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:39.040193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:39.041426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:39.042432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 431/858] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:51:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:39.052238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07a7d9-652f-4507-ad43-2af8c0cfd785 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:39.063857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b066cb-c218-47e0-8f62-1db05a7a1f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:39.064536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b066cb-c218-47e0-8f62-1db05a7a1f2c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:39.064824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b066cb-c218-47e0-8f62-1db05a7a1f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:39.065105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b066cb-c218-47e0-8f62-1db05a7a1f2c None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:39.065827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b066cb-c218-47e0-8f62-1db05a7a1f2c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:39.066135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b066cb-c218-47e0-8f62-1db05a7a1f2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:39.067734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b066cb-c218-47e0-8f62-1db05a7a1f2c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:39.082475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 428/859] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:39.161070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:39.180200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07a7d9-652f-4507-ad43-2af8c0cfd785 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:39.218085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7578da6-bcd2-4d74-a29a-fd6e0c722b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:39.218852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7578da6-bcd2-4d74-a29a-fd6e0c722b91 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:39.219143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7578da6-bcd2-4d74-a29a-fd6e0c722b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:39.219421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7578da6-bcd2-4d74-a29a-fd6e0c722b91 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:39.220187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7578da6-bcd2-4d74-a29a-fd6e0c722b91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:39.220496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7578da6-bcd2-4d74-a29a-fd6e0c722b91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:39.221901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7578da6-bcd2-4d74-a29a-fd6e0c722b91 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:39.231787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 432/860] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:51:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:39.239558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b066cb-c218-47e0-8f62-1db05a7a1f2c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:39.261368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:39.351625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b066cb-c218-47e0-8f62-1db05a7a1f2c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:39.387361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb0ae-0f09-4cd3-a474-87381c8f99d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:39.388154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb0ae-0f09-4cd3-a474-87381c8f99d9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:39.388440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb0ae-0f09-4cd3-a474-87381c8f99d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:39.388718 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb0ae-0f09-4cd3-a474-87381c8f99d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:39.400218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efb0ae-0f09-4cd3-a474-87381c8f99d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:39.401676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efb0ae-0f09-4cd3-a474-87381c8f99d9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-13efb0ae-0f09-4cd3-a474-87381c8f99d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:39.419026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:39.419945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:39.420844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:39.420844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:39.420844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:39.420844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:39.420844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 429/861] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:39.427093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7578da6-bcd2-4d74-a29a-fd6e0c722b91 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:39.529169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7578da6-bcd2-4d74-a29a-fd6e0c722b91 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:39.569806 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00655dc8-81a7-45e6-84f8-2551d4e4dd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:39.570522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00655dc8-81a7-45e6-84f8-2551d4e4dd6b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:39.570811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00655dc8-81a7-45e6-84f8-2551d4e4dd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:39.571087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00655dc8-81a7-45e6-84f8-2551d4e4dd6b None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:39.571829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00655dc8-81a7-45e6-84f8-2551d4e4dd6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:39.572137 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00655dc8-81a7-45e6-84f8-2551d4e4dd6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:39.573563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00655dc8-81a7-45e6-84f8-2551d4e4dd6b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:39.587167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 433/862] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:39.592522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb0ae-0f09-4cd3-a474-87381c8f99d9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:39.666921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:39.699529 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb0ae-0f09-4cd3-a474-87381c8f99d9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:39.734218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0b66-e90b-4afd-81ac-b23cceccac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:39.735254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0b66-e90b-4afd-81ac-b23cceccac01 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:39.735541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0b66-e90b-4afd-81ac-b23cceccac01 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:39.735821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0b66-e90b-4afd-81ac-b23cceccac01 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:39.744670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc0b66-e90b-4afd-81ac-b23cceccac01 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:39.746213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc0b66-e90b-4afd-81ac-b23cceccac01 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-edcc0b66-e90b-4afd-81ac-b23cceccac01 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:39.761273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:39.762239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:39.763226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 430/863] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 26 20:51:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:39.768435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00655dc8-81a7-45e6-84f8-2551d4e4dd6b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:39.790808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:39.872133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00655dc8-81a7-45e6-84f8-2551d4e4dd6b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:39.906511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351b520-177b-4363-b6d7-87cb18da6d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:39.907187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b520-177b-4363-b6d7-87cb18da6d86 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:39.907468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b520-177b-4363-b6d7-87cb18da6d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:39.907743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351b520-177b-4363-b6d7-87cb18da6d86 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:39.908567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351b520-177b-4363-b6d7-87cb18da6d86 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:39.908874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351b520-177b-4363-b6d7-87cb18da6d86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:39.910303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8351b520-177b-4363-b6d7-87cb18da6d86 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:39.928148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 434/864] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 26 20:51:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:39.933377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0b66-e90b-4afd-81ac-b23cceccac01 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:40.041653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0b66-e90b-4afd-81ac-b23cceccac01 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:40.068187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:40.077129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937e42c-dd30-4853-8162-055cec046e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:40.077782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937e42c-dd30-4853-8162-055cec046e13 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:40.078065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937e42c-dd30-4853-8162-055cec046e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:40.078347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937e42c-dd30-4853-8162-055cec046e13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:40.151626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937e42c-dd30-4853-8162-055cec046e13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:40.153111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937e42c-dd30-4853-8162-055cec046e13 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:40.258112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 431/865] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:51:40.264034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351b520-177b-4363-b6d7-87cb18da6d86 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:40.327841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:40.366534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351b520-177b-4363-b6d7-87cb18da6d86 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:40.402004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b1ea7-faf9-48cc-9345-1394cf102aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:40.402853 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b1ea7-faf9-48cc-9345-1394cf102aae None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:40.403135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b1ea7-faf9-48cc-9345-1394cf102aae None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:40.403408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b1ea7-faf9-48cc-9345-1394cf102aae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:40.413862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b1ea7-faf9-48cc-9345-1394cf102aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:40.415329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b1ea7-faf9-48cc-9345-1394cf102aae None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-106b1ea7-faf9-48cc-9345-1394cf102aae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:40.432667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:40.433556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:40.434497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:40.434497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:40.434497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:40.434497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:40.434497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 435/866] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:40.440922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937e42c-dd30-4853-8162-055cec046e13 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:40.471134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:40.552228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937e42c-dd30-4853-8162-055cec046e13 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:40.589390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af45e0-4faa-4b67-8631-0bffb9e107fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:40.590410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af45e0-4faa-4b67-8631-0bffb9e107fc None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:40.590932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af45e0-4faa-4b67-8631-0bffb9e107fc None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:40.591428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af45e0-4faa-4b67-8631-0bffb9e107fc None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:40.592364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af45e0-4faa-4b67-8631-0bffb9e107fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:40.592871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af45e0-4faa-4b67-8631-0bffb9e107fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:40.594459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af45e0-4faa-4b67-8631-0bffb9e107fc None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:40.606777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 432/867] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:40.612440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b1ea7-faf9-48cc-9345-1394cf102aae None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:40.695139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d124f36-79cc-4404-8488-aef5420805c1 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:40.695810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d124f36-79cc-4404-8488-aef5420805c1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:40.696291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d124f36-79cc-4404-8488-aef5420805c1 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:40.698102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 433/868] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:40.707295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3d44c-9aa9-4f33-a280-eff158a804a6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:40.707586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3d44c-9aa9-4f33-a280-eff158a804a6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:40.707864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3d44c-9aa9-4f33-a280-eff158a804a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:40.727439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e3d3d44c-9aa9-4f33-a280-eff158a804a6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:40.727969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b1ea7-faf9-48cc-9345-1394cf102aae None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:40.728615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e3d3d44c-9aa9-4f33-a280-eff158a804a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:40.748138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:40.769566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515910f4-4e1e-48bf-8bc8-cd92723f08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:40.770722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515910f4-4e1e-48bf-8bc8-cd92723f08a2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:40.771208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515910f4-4e1e-48bf-8bc8-cd92723f08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:40.771681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515910f4-4e1e-48bf-8bc8-cd92723f08a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:40.779271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515910f4-4e1e-48bf-8bc8-cd92723f08a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:40.781262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515910f4-4e1e-48bf-8bc8-cd92723f08a2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-515910f4-4e1e-48bf-8bc8-cd92723f08a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:40.796629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:40.797563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:40.798503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:40.798503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:40.798503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:40.798503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:40.798503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:40.798503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:40.798503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:40.798503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:40.804190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 436/869] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:40.808085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 434/870] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:40.810480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515910f4-4e1e-48bf-8bc8-cd92723f08a2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:40.836541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3d44c-9aa9-4f33-a280-eff158a804a6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:40.859871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:40.934540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515910f4-4e1e-48bf-8bc8-cd92723f08a2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:40.948928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3d44c-9aa9-4f33-a280-eff158a804a6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:40.972666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81dbd47-a329-4a03-90fc-00a416ebd1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:40.973944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dbd47-a329-4a03-90fc-00a416ebd1ec None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:40.974374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dbd47-a329-4a03-90fc-00a416ebd1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:40.974815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81dbd47-a329-4a03-90fc-00a416ebd1ec None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:40.976353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81dbd47-a329-4a03-90fc-00a416ebd1ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:40.976840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81dbd47-a329-4a03-90fc-00a416ebd1ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:40.978615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81dbd47-a329-4a03-90fc-00a416ebd1ec None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:40.985236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b46be8-b93d-4902-9c80-b3b591f4b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:40.986023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b46be8-b93d-4902-9c80-b3b591f4b6ad None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:40.986407 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b46be8-b93d-4902-9c80-b3b591f4b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:40.986810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b46be8-b93d-4902-9c80-b3b591f4b6ad None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:40.987702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b46be8-b93d-4902-9c80-b3b591f4b6ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:40.989120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b46be8-b93d-4902-9c80-b3b591f4b6ad None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:40.997871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 437/871] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:41.000692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81dbd47-a329-4a03-90fc-00a416ebd1ec None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:41.011336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 435/872] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:51:40 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:51:41.048801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:41.125875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81dbd47-a329-4a03-90fc-00a416ebd1ec None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:41.169716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d6e86-4bfa-468d-86b5-75fbac80ef89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:41.170388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d6e86-4bfa-468d-86b5-75fbac80ef89 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:41.170704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d6e86-4bfa-468d-86b5-75fbac80ef89 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:41.170979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d6e86-4bfa-468d-86b5-75fbac80ef89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:41.171811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d6e86-4bfa-468d-86b5-75fbac80ef89 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=e67f497e8a8b46e1b665999ba8fda38d) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:41.172129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d6e86-4bfa-468d-86b5-75fbac80ef89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:41.173575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d6e86-4bfa-468d-86b5-75fbac80ef89 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:41.181329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 438/873] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:40 2022] GET /identity/v3/role_assignments?user.id=e67f497e8a8b46e1b665999ba8fda38d&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:41.290375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e80c1-5bac-4ced-ad74-2200adeaf355 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:41.290719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e80c1-5bac-4ced-ad74-2200adeaf355 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:41.290971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e80c1-5bac-4ced-ad74-2200adeaf355 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:41.292382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 436/874] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:41.298292 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108222a-944b-404d-a069-28d46a0bd30c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:41.298617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108222a-944b-404d-a069-28d46a0bd30c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:41.298901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108222a-944b-404d-a069-28d46a0bd30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:41.317502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c108222a-944b-404d-a069-28d46a0bd30c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:41.318756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c108222a-944b-404d-a069-28d46a0bd30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:41.396759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 439/875] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:41.406710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a079b-844c-498f-b1e5-366175cfbbae None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:41.407001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a079b-844c-498f-b1e5-366175cfbbae None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:41.407284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a079b-844c-498f-b1e5-366175cfbbae None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:41.408875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 437/876] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:41.414589 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ce9b5-86f8-476c-882d-acb649756989 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:41.414862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ce9b5-86f8-476c-882d-acb649756989 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:41.415141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ce9b5-86f8-476c-882d-acb649756989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:41.422085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a079b-844c-498f-b1e5-366175cfbbae None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:41.432103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5e9ce9b5-86f8-476c-882d-acb649756989 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:41.433300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5e9ce9b5-86f8-476c-882d-acb649756989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:41.459017 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:41.517468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 440/877] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:41.534816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:41.540974 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ce9b5-86f8-476c-882d-acb649756989 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:41.568255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a079b-844c-498f-b1e5-366175cfbbae None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:41.608833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c779d-7c23-4003-86cc-e54241f34b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:41.610266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c779d-7c23-4003-86cc-e54241f34b47 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:41.610697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c779d-7c23-4003-86cc-e54241f34b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:41.610976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c779d-7c23-4003-86cc-e54241f34b47 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:41.611932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c779d-7c23-4003-86cc-e54241f34b47 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:41.614038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c779d-7c23-4003-86cc-e54241f34b47 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-320c779d-7c23-4003-86cc-e54241f34b47 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:41.630029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:41.631316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:41.632418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 438/878] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:41.638004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c779d-7c23-4003-86cc-e54241f34b47 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:41.658633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ce9b5-86f8-476c-882d-acb649756989 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:41.696984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cc7b6-e854-4140-96ee-defd032fdcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:41.696984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cc7b6-e854-4140-96ee-defd032fdcf0 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:41.697527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cc7b6-e854-4140-96ee-defd032fdcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:41.697884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cc7b6-e854-4140-96ee-defd032fdcf0 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:41.698716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cc7b6-e854-4140-96ee-defd032fdcf0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:41.700146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cc7b6-e854-4140-96ee-defd032fdcf0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:41.722543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 441/879] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:51:41 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:51:41.756174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c779d-7c23-4003-86cc-e54241f34b47 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:41.801002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091cf5f-85d8-463d-a79f-b4716d53f8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:41.804886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091cf5f-85d8-463d-a79f-b4716d53f8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:41.804886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091cf5f-85d8-463d-a79f-b4716d53f8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:41.804886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091cf5f-85d8-463d-a79f-b4716d53f8c4 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:41.804886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091cf5f-85d8-463d-a79f-b4716d53f8c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:41.804886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091cf5f-85d8-463d-a79f-b4716d53f8c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:41.804886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9091cf5f-85d8-463d-a79f-b4716d53f8c4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:41.813840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 439/880] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 26 20:51:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:41.822472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cc7b6-e854-4140-96ee-defd032fdcf0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:41.839666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:41.936284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cc7b6-e854-4140-96ee-defd032fdcf0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:41.971611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40692cd4-17b4-4363-9384-0f345849f80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:41.972277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40692cd4-17b4-4363-9384-0f345849f80e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:41.972562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40692cd4-17b4-4363-9384-0f345849f80e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:41.972842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40692cd4-17b4-4363-9384-0f345849f80e None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:41.973544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40692cd4-17b4-4363-9384-0f345849f80e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:41.973853 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40692cd4-17b4-4363-9384-0f345849f80e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:41.975251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40692cd4-17b4-4363-9384-0f345849f80e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:41.980171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 442/881] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 26 20:51:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:43.353691 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe854ef0-df65-4020-9165-8983a2a103bf None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:43.354274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe854ef0-df65-4020-9165-8983a2a103bf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:43.354389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe854ef0-df65-4020-9165-8983a2a103bf None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:43.356650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 440/882] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:43.362946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842f9d1-5568-4df6-9ef6-9a1a7fc93f5a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:43.363797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842f9d1-5568-4df6-9ef6-9a1a7fc93f5a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:43.364302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842f9d1-5568-4df6-9ef6-9a1a7fc93f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:43.382742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1842f9d1-5568-4df6-9ef6-9a1a7fc93f5a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:43.384358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1842f9d1-5568-4df6-9ef6-9a1a7fc93f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:43.466553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 443/883] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:43.484832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe854ef0-df65-4020-9165-8983a2a103bf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:43.488134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:43.580768 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d885-df34-49bd-8409-5eb1c5db6137 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:43.581058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d885-df34-49bd-8409-5eb1c5db6137 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:43.581576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64d885-df34-49bd-8409-5eb1c5db6137 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:43.583365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 444/884] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:43.589327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fa281-8f01-4b93-9355-2c7b144f8427 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:43.589898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fa281-8f01-4b93-9355-2c7b144f8427 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:43.590391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3fa281-8f01-4b93-9355-2c7b144f8427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:43.597052 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe854ef0-df65-4020-9165-8983a2a103bf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:43.608659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-de3fa281-8f01-4b93-9355-2c7b144f8427 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:43.610130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-de3fa281-8f01-4b93-9355-2c7b144f8427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:43.613553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:43.633467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7330cea-8532-4bd5-a695-798a8941f577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:43.635432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7330cea-8532-4bd5-a695-798a8941f577 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:43.635944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7330cea-8532-4bd5-a695-798a8941f577 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:43.636682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7330cea-8532-4bd5-a695-798a8941f577 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:43.637811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7330cea-8532-4bd5-a695-798a8941f577 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:43.640150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7330cea-8532-4bd5-a695-798a8941f577 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d7330cea-8532-4bd5-a695-798a8941f577 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:43.656696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:43.657812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:51:43.658776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:43.660432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 441/885] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:51:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:43.668377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7330cea-8532-4bd5-a695-798a8941f577 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:43.694111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 445/886] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:43.719595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fa281-8f01-4b93-9355-2c7b144f8427 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:43.757487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:43.798309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7330cea-8532-4bd5-a695-798a8941f577 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:43.837179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39362bc6-12b4-4cd3-91fe-2a298afffc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:43.837735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3fa281-8f01-4b93-9355-2c7b144f8427 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:43.838312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39362bc6-12b4-4cd3-91fe-2a298afffc9a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:43.838784 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39362bc6-12b4-4cd3-91fe-2a298afffc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:43.839244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39362bc6-12b4-4cd3-91fe-2a298afffc9a None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:43.840345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39362bc6-12b4-4cd3-91fe-2a298afffc9a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:43.840930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39362bc6-12b4-4cd3-91fe-2a298afffc9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:43.842650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39362bc6-12b4-4cd3-91fe-2a298afffc9a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:43.853736 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 442/887] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:51:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:43.855741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39362bc6-12b4-4cd3-91fe-2a298afffc9a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:43.879294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14443bc-0c4f-4203-b258-646715c4dee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:43.880493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14443bc-0c4f-4203-b258-646715c4dee3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:43.880993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14443bc-0c4f-4203-b258-646715c4dee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:43.881539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14443bc-0c4f-4203-b258-646715c4dee3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:43.891218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:43.895134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14443bc-0c4f-4203-b258-646715c4dee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-f14443bc-0c4f-4203-b258-646715c4dee3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:43.904500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:43.905568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:51:43.906429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:43.913830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 446/888] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 26 20:51:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:43.914979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14443bc-0c4f-4203-b258-646715c4dee3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:43.973061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39362bc6-12b4-4cd3-91fe-2a298afffc9a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.004812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:44.013071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25791cc2-041b-4086-952f-578187342104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:44.013860 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25791cc2-041b-4086-952f-578187342104 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.014152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25791cc2-041b-4086-952f-578187342104 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.014436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25791cc2-041b-4086-952f-578187342104 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.015405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25791cc2-041b-4086-952f-578187342104 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:44.015602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25791cc2-041b-4086-952f-578187342104 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:44.017022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25791cc2-041b-4086-952f-578187342104 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:44.022077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14443bc-0c4f-4203-b258-646715c4dee3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.028092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 443/889] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:51:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:44.029579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25791cc2-041b-4086-952f-578187342104 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:44.064560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1836ba-a1ef-49b4-826c-30bfb3df22b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:44.066245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1836ba-a1ef-49b4-826c-30bfb3df22b9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.067065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1836ba-a1ef-49b4-826c-30bfb3df22b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.067607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1836ba-a1ef-49b4-826c-30bfb3df22b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.068619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1836ba-a1ef-49b4-826c-30bfb3df22b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:44.069430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1836ba-a1ef-49b4-826c-30bfb3df22b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:44.071703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1836ba-a1ef-49b4-826c-30bfb3df22b9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:44.084525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 447/890] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 26 20:51:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:44.084525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f67de-880c-4193-8508-227d1fe23ebd None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.084525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f67de-880c-4193-8508-227d1fe23ebd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.084525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f67de-880c-4193-8508-227d1fe23ebd None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.084525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 448/891] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:44.087950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f67de-880c-4193-8508-227d1fe23ebd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:44.146920 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25791cc2-041b-4086-952f-578187342104 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.150991 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:44.184666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b5473-39dd-4e58-8c2a-1803358c874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:44.185325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b5473-39dd-4e58-8c2a-1803358c874f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.185612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b5473-39dd-4e58-8c2a-1803358c874f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.185890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b5473-39dd-4e58-8c2a-1803358c874f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.186708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b5473-39dd-4e58-8c2a-1803358c874f None admin] RBAC: Adding query filter params (interface=public, service_id=2f586d29d1da41c79cc98281c5e35bcd, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:44.187025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b5473-39dd-4e58-8c2a-1803358c874f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:44.188409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b5473-39dd-4e58-8c2a-1803358c874f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:44.198159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 444/892] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 26 20:51:44 2022] GET /identity/v3/endpoints?service_id=2f586d29d1da41c79cc98281c5e35bcd&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:44.198159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d410-e92c-43a6-ac53-e731dcd12b56 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.198159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d410-e92c-43a6-ac53-e731dcd12b56 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.198159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654d410-e92c-43a6-ac53-e731dcd12b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.199514 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f67de-880c-4193-8508-227d1fe23ebd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.213585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6654d410-e92c-43a6-ac53-e731dcd12b56 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:44.214785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6654d410-e92c-43a6-ac53-e731dcd12b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:44.234281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7ee3c-f5a6-43ff-83de-80cb1de98f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:44.235111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ee3c-f5a6-43ff-83de-80cb1de98f2a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.235398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ee3c-f5a6-43ff-83de-80cb1de98f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.235680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ee3c-f5a6-43ff-83de-80cb1de98f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.247323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7ee3c-f5a6-43ff-83de-80cb1de98f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:44.248760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7ee3c-f5a6-43ff-83de-80cb1de98f2a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-cdb7ee3c-f5a6-43ff-83de-80cb1de98f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:44.266286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:44.267439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:44.268480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:44.268480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:44.268480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:44.268480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:44.268480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 449/893] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:44.268480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7ee3c-f5a6-43ff-83de-80cb1de98f2a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:44.272662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:44.284045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 445/894] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:44.285342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40680ed-7a92-46ee-bce6-43a7ac030df8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.285634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40680ed-7a92-46ee-bce6-43a7ac030df8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.285915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40680ed-7a92-46ee-bce6-43a7ac030df8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.287451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 446/895] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:44.288672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40680ed-7a92-46ee-bce6-43a7ac030df8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:44.371389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7ee3c-f5a6-43ff-83de-80cb1de98f2a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.384437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:44.400359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40680ed-7a92-46ee-bce6-43a7ac030df8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.405483 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88db44c-5163-4bca-92a7-3ac9bb1b210a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:44.406145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88db44c-5163-4bca-92a7-3ac9bb1b210a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.406428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88db44c-5163-4bca-92a7-3ac9bb1b210a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.406733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88db44c-5163-4bca-92a7-3ac9bb1b210a None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.407410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88db44c-5163-4bca-92a7-3ac9bb1b210a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:44.408771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88db44c-5163-4bca-92a7-3ac9bb1b210a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:44.415152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 450/896] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 26 20:51:44 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:51:44.416234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f835e-c969-4c1e-81bc-d0f6013a52df None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.416526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f835e-c969-4c1e-81bc-d0f6013a52df None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.416811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f835e-c969-4c1e-81bc-d0f6013a52df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.435583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-624f835e-c969-4c1e-81bc-d0f6013a52df None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:44.436838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-624f835e-c969-4c1e-81bc-d0f6013a52df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:44.437502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd343acc-bfa3-4b34-91e4-16ec97c4b3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:44.438187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd343acc-bfa3-4b34-91e4-16ec97c4b3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.438495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd343acc-bfa3-4b34-91e4-16ec97c4b3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.438776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd343acc-bfa3-4b34-91e4-16ec97c4b3b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.439663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd343acc-bfa3-4b34-91e4-16ec97c4b3b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:44.440823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd343acc-bfa3-4b34-91e4-16ec97c4b3b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:44.441487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd343acc-bfa3-4b34-91e4-16ec97c4b3b0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:44.459516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 447/897] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:44.459516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd343acc-bfa3-4b34-91e4-16ec97c4b3b0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:44.515119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:44.517522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 451/898] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:44.518200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f835e-c969-4c1e-81bc-d0f6013a52df None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:44.592413 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd343acc-bfa3-4b34-91e4-16ec97c4b3b0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.630543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe00f43-0c1d-4cd9-b41f-736cb20acf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:44.631367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe00f43-0c1d-4cd9-b41f-736cb20acf90 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.631682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe00f43-0c1d-4cd9-b41f-736cb20acf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.631876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f835e-c969-4c1e-81bc-d0f6013a52df None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.631996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe00f43-0c1d-4cd9-b41f-736cb20acf90 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.632616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe00f43-0c1d-4cd9-b41f-736cb20acf90 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:44.633993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe00f43-0c1d-4cd9-b41f-736cb20acf90 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:44.642684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5fe00f43-0c1d-4cd9-b41f-736cb20acf90 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:44.646639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:44.648139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:44.648977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 448/899] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 26 20:51:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:44.649831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe00f43-0c1d-4cd9-b41f-736cb20acf90 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:44.679381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56a92c-d3ea-4b25-9bfe-6139f1979831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:44.680450 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56a92c-d3ea-4b25-9bfe-6139f1979831 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.680737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56a92c-d3ea-4b25-9bfe-6139f1979831 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.681014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56a92c-d3ea-4b25-9bfe-6139f1979831 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.689352 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56a92c-d3ea-4b25-9bfe-6139f1979831 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:44.691172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56a92c-d3ea-4b25-9bfe-6139f1979831 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1f56a92c-d3ea-4b25-9bfe-6139f1979831 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:44.708149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:44.709168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 452/900] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 26 20:51:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:44.711414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56a92c-d3ea-4b25-9bfe-6139f1979831 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:44.754403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe00f43-0c1d-4cd9-b41f-736cb20acf90 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.769292 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:44.792330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45de3-5dcd-491d-bf54-85b0480664c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:44.792999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45de3-5dcd-491d-bf54-85b0480664c7 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.793279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45de3-5dcd-491d-bf54-85b0480664c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.793564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45de3-5dcd-491d-bf54-85b0480664c7 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.794290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c45de3-5dcd-491d-bf54-85b0480664c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:44.795713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c45de3-5dcd-491d-bf54-85b0480664c7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:44.815560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 449/901] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:51:44 2022] POST /identity/v3/services => generated 241 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:51:44.816261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45de3-5dcd-491d-bf54-85b0480664c7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:44.838074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56a92c-d3ea-4b25-9bfe-6139f1979831 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.905258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad2e43-e6a1-4a26-8ac1-8eeacbdcfd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:44.907539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad2e43-e6a1-4a26-8ac1-8eeacbdcfd47 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:44.908051 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad2e43-e6a1-4a26-8ac1-8eeacbdcfd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:44.908518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad2e43-e6a1-4a26-8ac1-8eeacbdcfd47 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:44.909466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad2e43-e6a1-4a26-8ac1-8eeacbdcfd47 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:44.909995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad2e43-e6a1-4a26-8ac1-8eeacbdcfd47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:44.911686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad2e43-e6a1-4a26-8ac1-8eeacbdcfd47 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:44.924222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 453/902] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:44.926329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad2e43-e6a1-4a26-8ac1-8eeacbdcfd47 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:44.947980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45de3-5dcd-491d-bf54-85b0480664c7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:44.969487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:44.999693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41569-7e0d-4870-9430-e1c8d74627c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:45.000610 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41569-7e0d-4870-9430-e1c8d74627c0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:45.000896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41569-7e0d-4870-9430-e1c8d74627c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:45.001175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f41569-7e0d-4870-9430-e1c8d74627c0 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:45.002026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f41569-7e0d-4870-9430-e1c8d74627c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:45.002341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f41569-7e0d-4870-9430-e1c8d74627c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:45.003838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f41569-7e0d-4870-9430-e1c8d74627c0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:45.025913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 450/903] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 26 20:51:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:45.032652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41569-7e0d-4870-9430-e1c8d74627c0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:45.063908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad2e43-e6a1-4a26-8ac1-8eeacbdcfd47 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:45.075907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:45.103981 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9ec40-47c7-4ab0-9685-80e1363a7e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:45.104755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9ec40-47c7-4ab0-9685-80e1363a7e9c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:45.105049 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9ec40-47c7-4ab0-9685-80e1363a7e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:45.105332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9ec40-47c7-4ab0-9685-80e1363a7e9c None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:45.106046 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9ec40-47c7-4ab0-9685-80e1363a7e9c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:45.106481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9ec40-47c7-4ab0-9685-80e1363a7e9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:45.107967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9ec40-47c7-4ab0-9685-80e1363a7e9c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:45.118471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 454/904] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:45.119756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9ec40-47c7-4ab0-9685-80e1363a7e9c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:45.151687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f41569-7e0d-4870-9430-e1c8d74627c0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:45.195327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2622b-de20-48f6-864c-49f4a74157c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:45.196015 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2622b-de20-48f6-864c-49f4a74157c5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:45.196588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2622b-de20-48f6-864c-49f4a74157c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:45.196697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca2622b-de20-48f6-864c-49f4a74157c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:45.234359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:45.245959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9ec40-47c7-4ab0-9685-80e1363a7e9c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:45.271675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2622b-de20-48f6-864c-49f4a74157c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:45.273294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca2622b-de20-48f6-864c-49f4a74157c5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:45.285804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fe655-733e-48a2-8edb-7afdaa234941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:45.286858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fe655-733e-48a2-8edb-7afdaa234941 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:45.287341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fe655-733e-48a2-8edb-7afdaa234941 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:45.287969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fe655-733e-48a2-8edb-7afdaa234941 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:45.289057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fe655-733e-48a2-8edb-7afdaa234941 None admin] RBAC: Adding query filter params (interface=public, service_id=5578ba23e007451a95508de66ac57a45, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:45.289624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fe655-733e-48a2-8edb-7afdaa234941 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:45.291305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fe655-733e-48a2-8edb-7afdaa234941 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:45.299626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 455/905] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 26 20:51:45 2022] GET /identity/v3/endpoints?service_id=5578ba23e007451a95508de66ac57a45&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:45.303363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:45.306099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fe655-733e-48a2-8edb-7afdaa234941 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:45.384315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 451/906] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:45 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 20:51:45.390074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2622b-de20-48f6-864c-49f4a74157c5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:45.439400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fe655-733e-48a2-8edb-7afdaa234941 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:45.486378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214aaff-d0ad-490b-a5e8-caa73c62e0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:45.487542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214aaff-d0ad-490b-a5e8-caa73c62e0eb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:45.488212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214aaff-d0ad-490b-a5e8-caa73c62e0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:45.488650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214aaff-d0ad-490b-a5e8-caa73c62e0eb None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:45.489874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214aaff-d0ad-490b-a5e8-caa73c62e0eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:45.492476 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214aaff-d0ad-490b-a5e8-caa73c62e0eb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:45.495811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca2622b-de20-48f6-864c-49f4a74157c5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:45.501327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 456/907] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 26 20:51:45 2022] GET /identity/v3/services => generated 1497 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:51:45.534159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:45.543172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ee7f2-4d66-4fa6-afe8-5df0919b3719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:45.544060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ee7f2-4d66-4fa6-afe8-5df0919b3719 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:45.544358 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ee7f2-4d66-4fa6-afe8-5df0919b3719 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:45.544671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ee7f2-4d66-4fa6-afe8-5df0919b3719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:45.557201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ee7f2-4d66-4fa6-afe8-5df0919b3719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:45.558752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ee7f2-4d66-4fa6-afe8-5df0919b3719 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b67ee7f2-4d66-4fa6-afe8-5df0919b3719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:45.581150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:45.582121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:45.583048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:45.583048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:45.583048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:45.583048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:45.583048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 452/908] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:45.589543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214aaff-d0ad-490b-a5e8-caa73c62e0eb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:45.620274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:45.711166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214aaff-d0ad-490b-a5e8-caa73c62e0eb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:45.747627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da295e-d47d-408d-b867-9779841b9c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:45.748224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da295e-d47d-408d-b867-9779841b9c1d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:45.748512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da295e-d47d-408d-b867-9779841b9c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:45.748796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da295e-d47d-408d-b867-9779841b9c1d None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:45.749523 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da295e-d47d-408d-b867-9779841b9c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:45.749832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da295e-d47d-408d-b867-9779841b9c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:45.751235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da295e-d47d-408d-b867-9779841b9c1d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:45.764325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 457/909] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:45.769442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ee7f2-4d66-4fa6-afe8-5df0919b3719 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:45.871465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ee7f2-4d66-4fa6-afe8-5df0919b3719 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:45.906174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f12df-8e6e-491c-9579-f01186a9292e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:45.906998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f12df-8e6e-491c-9579-f01186a9292e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:45.907282 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f12df-8e6e-491c-9579-f01186a9292e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:45.907561 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f12df-8e6e-491c-9579-f01186a9292e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:45.914967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f12df-8e6e-491c-9579-f01186a9292e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:45.916427 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f12df-8e6e-491c-9579-f01186a9292e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fc7f12df-8e6e-491c-9579-f01186a9292e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:45.930213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:45.931132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:45.932034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:45.932034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:45.932034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:45.932034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:45.932034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:45.932034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:45.932034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:45.932034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:45.932034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 453/910] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:45.937403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da295e-d47d-408d-b867-9779841b9c1d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:46.022726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:46.043128 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da295e-d47d-408d-b867-9779841b9c1d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:46.080994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1537b-9521-47d5-9d20-76dbd96f81b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:46.081664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1537b-9521-47d5-9d20-76dbd96f81b0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:46.081948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1537b-9521-47d5-9d20-76dbd96f81b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:46.082227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1537b-9521-47d5-9d20-76dbd96f81b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:46.083044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1537b-9521-47d5-9d20-76dbd96f81b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:46.083354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1537b-9521-47d5-9d20-76dbd96f81b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:46.085327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1537b-9521-47d5-9d20-76dbd96f81b0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:46.097819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 458/911] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:46.104304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f12df-8e6e-491c-9579-f01186a9292e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:46.150258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:46.208989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f12df-8e6e-491c-9579-f01186a9292e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:46.243696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dc427-33c4-49f1-b436-4bb7231d7023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:46.244356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dc427-33c4-49f1-b436-4bb7231d7023 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:46.244642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dc427-33c4-49f1-b436-4bb7231d7023 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:46.244917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dc427-33c4-49f1-b436-4bb7231d7023 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:46.245747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dc427-33c4-49f1-b436-4bb7231d7023 None admin] RBAC: Adding query filter params (role.id=190ab872a79449cdbc3d501f21d56d63, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=e67f497e8a8b46e1b665999ba8fda38d) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:46.246059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dc427-33c4-49f1-b436-4bb7231d7023 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:46.247544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dc427-33c4-49f1-b436-4bb7231d7023 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:46.253540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 454/912] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:46 2022] GET /identity/v3/role_assignments?user.id=e67f497e8a8b46e1b665999ba8fda38d&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=190ab872a79449cdbc3d501f21d56d63 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:46.655244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fb2bb-c5da-43c8-a5e3-550111d09619 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:46.655676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fb2bb-c5da-43c8-a5e3-550111d09619 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:46.655821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fb2bb-c5da-43c8-a5e3-550111d09619 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:46.657427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 459/913] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:46.662824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0607f8-2542-402e-acad-4f9c730eee81 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:46.663224 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0607f8-2542-402e-acad-4f9c730eee81 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:46.663385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0607f8-2542-402e-acad-4f9c730eee81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:46.681466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0e0607f8-2542-402e-acad-4f9c730eee81 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:46.682729 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0e0607f8-2542-402e-acad-4f9c730eee81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:46.764365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 455/914] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:46.786242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fb2bb-c5da-43c8-a5e3-550111d09619 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:46.908191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0fb2bb-c5da-43c8-a5e3-550111d09619 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:46.943335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbadd8-7445-46f3-8d98-cb788b6b8565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:46.944683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbadd8-7445-46f3-8d98-cb788b6b8565 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:46.944973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbadd8-7445-46f3-8d98-cb788b6b8565 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:46.945254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbadd8-7445-46f3-8d98-cb788b6b8565 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:46.945908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedbadd8-7445-46f3-8d98-cb788b6b8565 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:46.947440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedbadd8-7445-46f3-8d98-cb788b6b8565 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-eedbadd8-7445-46f3-8d98-cb788b6b8565 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:46.960338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:46.961351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:46.962213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 460/915] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:51:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:46.969578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0607f8-2542-402e-acad-4f9c730eee81 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:47.009015 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:47.082562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0607f8-2542-402e-acad-4f9c730eee81 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:47.118505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69208b7b-1e7b-471c-8657-98e91b201275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:47.119182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69208b7b-1e7b-471c-8657-98e91b201275 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.119470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69208b7b-1e7b-471c-8657-98e91b201275 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.119754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69208b7b-1e7b-471c-8657-98e91b201275 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.120468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69208b7b-1e7b-471c-8657-98e91b201275 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:47.120781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69208b7b-1e7b-471c-8657-98e91b201275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:47.122175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69208b7b-1e7b-471c-8657-98e91b201275 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:47.127674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 456/916] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:51:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:47.133675 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbadd8-7445-46f3-8d98-cb788b6b8565 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:47.139129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:47.171941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7d7bc-1d8c-44fd-97a0-9d0031dbf045 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.172239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7d7bc-1d8c-44fd-97a0-9d0031dbf045 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.172417 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7d7bc-1d8c-44fd-97a0-9d0031dbf045 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.174019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 457/917] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:47.179179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157bef3-efff-466b-9051-99a104ac3bca None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.179570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157bef3-efff-466b-9051-99a104ac3bca None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.179948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157bef3-efff-466b-9051-99a104ac3bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.197807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9157bef3-efff-466b-9051-99a104ac3bca None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:47.199273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9157bef3-efff-466b-9051-99a104ac3bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:47.245886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbadd8-7445-46f3-8d98-cb788b6b8565 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:47.274798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 458/918] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:47.291220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157bef3-efff-466b-9051-99a104ac3bca None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:47.291364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5939af-ee2a-45bb-a7cf-8139714aaa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:47.292026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5939af-ee2a-45bb-a7cf-8139714aaa89 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.292318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5939af-ee2a-45bb-a7cf-8139714aaa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.292601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5939af-ee2a-45bb-a7cf-8139714aaa89 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.293313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5939af-ee2a-45bb-a7cf-8139714aaa89 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:47.293622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5939af-ee2a-45bb-a7cf-8139714aaa89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:47.295056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5939af-ee2a-45bb-a7cf-8139714aaa89 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:47.306304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 461/919] 10.222.0.57 () {56 vars in 1114 bytes} [Thu May 26 20:51:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:47.312736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5939af-ee2a-45bb-a7cf-8139714aaa89 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:47.350803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:47.383284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:47.413545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157bef3-efff-466b-9051-99a104ac3bca None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:47.435579 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5939af-ee2a-45bb-a7cf-8139714aaa89 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:47.456152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cc5f0-d823-4c41-b997-cfd1417ed49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:47.457305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cc5f0-d823-4c41-b997-cfd1417ed49e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.457815 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cc5f0-d823-4c41-b997-cfd1417ed49e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.458302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cc5f0-d823-4c41-b997-cfd1417ed49e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.459763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cc5f0-d823-4c41-b997-cfd1417ed49e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:47.461997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cc5f0-d823-4c41-b997-cfd1417ed49e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:47.474497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7454aa4-ce2c-4bc6-a820-1aab7f2e74c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:47.475178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7454aa4-ce2c-4bc6-a820-1aab7f2e74c6 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.475614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7454aa4-ce2c-4bc6-a820-1aab7f2e74c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.475904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7454aa4-ce2c-4bc6-a820-1aab7f2e74c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.476648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7454aa4-ce2c-4bc6-a820-1aab7f2e74c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:47.478021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7454aa4-ce2c-4bc6-a820-1aab7f2e74c6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-081cc5f0-d823-4c41-b997-cfd1417ed49e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:47.478526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:47.479523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:47.480390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:47.484148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 459/920] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:47.489386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cc5f0-d823-4c41-b997-cfd1417ed49e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:47.504323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 462/921] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 26 20:51:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:51:47.601308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:47.608833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cc5f0-d823-4c41-b997-cfd1417ed49e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:47.652990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3e847-854e-4cf9-9b54-ad9bbfdcef18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:47.653660 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3e847-854e-4cf9-9b54-ad9bbfdcef18 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.653943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3e847-854e-4cf9-9b54-ad9bbfdcef18 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.654221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3e847-854e-4cf9-9b54-ad9bbfdcef18 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.654974 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3e847-854e-4cf9-9b54-ad9bbfdcef18 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:47.655282 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3e847-854e-4cf9-9b54-ad9bbfdcef18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:47.656701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3e847-854e-4cf9-9b54-ad9bbfdcef18 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:47.662482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 460/922] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 26 20:51:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:47.668396 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7454aa4-ce2c-4bc6-a820-1aab7f2e74c6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:47.780599 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7454aa4-ce2c-4bc6-a820-1aab7f2e74c6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:47.807952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:47.823481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5bd6d-3702-48f1-9a54-a7e570984c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:47.824368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5bd6d-3702-48f1-9a54-a7e570984c79 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.824853 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5bd6d-3702-48f1-9a54-a7e570984c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.825318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5bd6d-3702-48f1-9a54-a7e570984c79 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.826225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5bd6d-3702-48f1-9a54-a7e570984c79 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:47.826749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5bd6d-3702-48f1-9a54-a7e570984c79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:47.828286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5bd6d-3702-48f1-9a54-a7e570984c79 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:47.833635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 463/923] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 26 20:51:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:47.839859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3e847-854e-4cf9-9b54-ad9bbfdcef18 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:47.887863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438567a-cb74-4d09-b465-7e40ce2ef947 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.888126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438567a-cb74-4d09-b465-7e40ce2ef947 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.888364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438567a-cb74-4d09-b465-7e40ce2ef947 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.889988 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 464/924] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:47.895191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64abbd2-c938-40d4-97d8-7eadd8a1af14 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.895384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64abbd2-c938-40d4-97d8-7eadd8a1af14 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.895619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64abbd2-c938-40d4-97d8-7eadd8a1af14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.916273 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b64abbd2-c938-40d4-97d8-7eadd8a1af14 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:47.917635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b64abbd2-c938-40d4-97d8-7eadd8a1af14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:47.949495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3e847-854e-4cf9-9b54-ad9bbfdcef18 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:47.991790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b53567-5cdb-4ec6-b27b-7ac642561012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:47.992460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b53567-5cdb-4ec6-b27b-7ac642561012 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:47.992748 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b53567-5cdb-4ec6-b27b-7ac642561012 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:47.993293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b53567-5cdb-4ec6-b27b-7ac642561012 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:47.994092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b53567-5cdb-4ec6-b27b-7ac642561012 None admin] RBAC: Adding query filter params (interface=public, service_id=3e14356ca9114885ab9ec88c8831dbdd, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:47.994409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b53567-5cdb-4ec6-b27b-7ac642561012 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:47.996765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b53567-5cdb-4ec6-b27b-7ac642561012 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:48.000585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 465/925] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:48.008494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 461/926] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 26 20:51:47 2022] GET /identity/v3/endpoints?service_id=3e14356ca9114885ab9ec88c8831dbdd&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:48.014364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64abbd2-c938-40d4-97d8-7eadd8a1af14 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:48.025346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b53567-5cdb-4ec6-b27b-7ac642561012 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:48.054890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:48.142410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b53567-5cdb-4ec6-b27b-7ac642561012 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:48.144171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64abbd2-c938-40d4-97d8-7eadd8a1af14 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:48.167590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:48.183923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf6300-bc82-4765-96dd-6c0c786efcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:48.185040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf6300-bc82-4765-96dd-6c0c786efcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:48.185542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf6300-bc82-4765-96dd-6c0c786efcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:48.185879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721991b-c3f0-41fe-9625-bd79a55d9e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:48.186273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf6300-bc82-4765-96dd-6c0c786efcc5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:48.186590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721991b-c3f0-41fe-9625-bd79a55d9e02 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:48.186878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721991b-c3f0-41fe-9625-bd79a55d9e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:48.187159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721991b-c3f0-41fe-9625-bd79a55d9e02 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:48.187785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf6300-bc82-4765-96dd-6c0c786efcc5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:48.187908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721991b-c3f0-41fe-9625-bd79a55d9e02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:48.189305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721991b-c3f0-41fe-9625-bd79a55d9e02 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:48.189661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf6300-bc82-4765-96dd-6c0c786efcc5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:48.202469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 466/927] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 26 20:51:48 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-30cf6300-bc82-4765-96dd-6c0c786efcc5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:48.206801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:48.207798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:48.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 462/928] 10.222.0.57 () {56 vars in 1132 bytes} [Thu May 26 20:51:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:48.215132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721991b-c3f0-41fe-9625-bd79a55d9e02 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:48.327938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721991b-c3f0-41fe-9625-bd79a55d9e02 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:48.369796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d1a23-beb0-4c58-b3c4-ad3248889f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:48.370893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d1a23-beb0-4c58-b3c4-ad3248889f87 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:48.371410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d1a23-beb0-4c58-b3c4-ad3248889f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:48.371887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d1a23-beb0-4c58-b3c4-ad3248889f87 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:48.372821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d1a23-beb0-4c58-b3c4-ad3248889f87 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:48.373337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d1a23-beb0-4c58-b3c4-ad3248889f87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:48.375020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d1a23-beb0-4c58-b3c4-ad3248889f87 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:48.381314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 467/929] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:48.387055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf6300-bc82-4765-96dd-6c0c786efcc5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:48.487001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:48.505758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf6300-bc82-4765-96dd-6c0c786efcc5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:48.541117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f030fe4-556b-4cce-9e53-576c7d898305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:48.541780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f030fe4-556b-4cce-9e53-576c7d898305 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:48.542063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f030fe4-556b-4cce-9e53-576c7d898305 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:48.542339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f030fe4-556b-4cce-9e53-576c7d898305 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:48.543079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f030fe4-556b-4cce-9e53-576c7d898305 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:48.543385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f030fe4-556b-4cce-9e53-576c7d898305 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:48.544761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f030fe4-556b-4cce-9e53-576c7d898305 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:48.550132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 463/930] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:48.555860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d1a23-beb0-4c58-b3c4-ad3248889f87 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:48.606502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:48.612368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ab634-96db-4bc8-8ca2-18b338819781 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:48.612670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ab634-96db-4bc8-8ca2-18b338819781 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:48.613016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ab634-96db-4bc8-8ca2-18b338819781 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:48.615749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 464/931] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:48.619946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330373-a451-4aab-abc1-ebf2965f5dee None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:48.620241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330373-a451-4aab-abc1-ebf2965f5dee None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:48.620525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45330373-a451-4aab-abc1-ebf2965f5dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:48.639068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-45330373-a451-4aab-abc1-ebf2965f5dee None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:48.640273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-45330373-a451-4aab-abc1-ebf2965f5dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:48.672190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d1a23-beb0-4c58-b3c4-ad3248889f87 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:48.719809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b962952-03dc-4596-9d31-ba4d167efb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:48.721298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b962952-03dc-4596-9d31-ba4d167efb38 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:48.721769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b962952-03dc-4596-9d31-ba4d167efb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:48.722361 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b962952-03dc-4596-9d31-ba4d167efb38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:48.723610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b962952-03dc-4596-9d31-ba4d167efb38 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:48.725443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b962952-03dc-4596-9d31-ba4d167efb38 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:48.727833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 465/932] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:48.753070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 468/933] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 26 20:51:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:51:48.754717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45330373-a451-4aab-abc1-ebf2965f5dee None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:48.852616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:48.880347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45330373-a451-4aab-abc1-ebf2965f5dee None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:48.925549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3c577-06bc-44d2-b0da-aa0581f76210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:48.926422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c577-06bc-44d2-b0da-aa0581f76210 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:48.926734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c577-06bc-44d2-b0da-aa0581f76210 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:48.927008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c577-06bc-44d2-b0da-aa0581f76210 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:48.942187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3c577-06bc-44d2-b0da-aa0581f76210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-88a3c577-06bc-44d2-b0da-aa0581f76210 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:48.951396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:48.952338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:51:48.953200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:48.955985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 466/934] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 26 20:51:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:48.959494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b962952-03dc-4596-9d31-ba4d167efb38 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:49.068468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b962952-03dc-4596-9d31-ba4d167efb38 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:49.070706 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:49.106731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc55936-7e48-441f-b00a-66182dfbfc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:49.107403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc55936-7e48-441f-b00a-66182dfbfc32 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:49.107693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc55936-7e48-441f-b00a-66182dfbfc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:49.107976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc55936-7e48-441f-b00a-66182dfbfc32 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:49.108721 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc55936-7e48-441f-b00a-66182dfbfc32 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:49.109034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc55936-7e48-441f-b00a-66182dfbfc32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:49.110408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc55936-7e48-441f-b00a-66182dfbfc32 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:49.124230 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 469/935] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 26 20:51:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:49.125560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3c577-06bc-44d2-b0da-aa0581f76210 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:49.231395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3c577-06bc-44d2-b0da-aa0581f76210 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:49.261722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:49.267117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d8473-58a5-409d-b878-c7ca0319f091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:49.267914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d8473-58a5-409d-b878-c7ca0319f091 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:49.268217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d8473-58a5-409d-b878-c7ca0319f091 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:49.268571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d8473-58a5-409d-b878-c7ca0319f091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:49.281082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d8473-58a5-409d-b878-c7ca0319f091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:49.282674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d8473-58a5-409d-b878-c7ca0319f091 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-927d8473-58a5-409d-b878-c7ca0319f091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:49.300805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:49.301806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:49.302741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:49.302741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:49.302741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:49.302741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:49.305771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 467/936] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:49.308620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc55936-7e48-441f-b00a-66182dfbfc32 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:49.414267 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc55936-7e48-441f-b00a-66182dfbfc32 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:49.452604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656c3cd-b60c-4e4c-b513-11b4bf968b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:49.453276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656c3cd-b60c-4e4c-b513-11b4bf968b2c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:49.453556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656c3cd-b60c-4e4c-b513-11b4bf968b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:49.453919 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656c3cd-b60c-4e4c-b513-11b4bf968b2c None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:49.454714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656c3cd-b60c-4e4c-b513-11b4bf968b2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:49.455022 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656c3cd-b60c-4e4c-b513-11b4bf968b2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:49.456389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656c3cd-b60c-4e4c-b513-11b4bf968b2c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:49.469720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 470/937] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:49.475206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d8473-58a5-409d-b878-c7ca0319f091 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:49.585837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d8473-58a5-409d-b878-c7ca0319f091 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:49.621303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbd2da-437c-4f99-9331-df828238a24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:49.622112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd2da-437c-4f99-9331-df828238a24f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:49.622394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd2da-437c-4f99-9331-df828238a24f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:49.622700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbd2da-437c-4f99-9331-df828238a24f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:49.630527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbd2da-437c-4f99-9331-df828238a24f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:49.632560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbd2da-437c-4f99-9331-df828238a24f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b8fbd2da-437c-4f99-9331-df828238a24f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:49.647478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:49.649594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:49.651025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 468/938] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 26 20:51:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:49.654464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656c3cd-b60c-4e4c-b513-11b4bf968b2c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:49.684871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:49.778079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656c3cd-b60c-4e4c-b513-11b4bf968b2c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:49.824588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2371d2f-f235-4eb7-8c33-e0625b56fa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:49.825540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2371d2f-f235-4eb7-8c33-e0625b56fa2b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:49.826020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2371d2f-f235-4eb7-8c33-e0625b56fa2b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:49.826526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2371d2f-f235-4eb7-8c33-e0625b56fa2b None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:49.827561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2371d2f-f235-4eb7-8c33-e0625b56fa2b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:49.828093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2371d2f-f235-4eb7-8c33-e0625b56fa2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:49.829703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2371d2f-f235-4eb7-8c33-e0625b56fa2b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:49.847804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 471/939] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 26 20:51:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:49.855802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbd2da-437c-4f99-9331-df828238a24f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:49.902120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088bf66-f225-4a42-9326-853d034f0326 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:49.902480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088bf66-f225-4a42-9326-853d034f0326 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:49.902715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088bf66-f225-4a42-9326-853d034f0326 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:49.904557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 472/940] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:49.909668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cf9be-3d3a-4785-94d2-d92516182de9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:49.910182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cf9be-3d3a-4785-94d2-d92516182de9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:49.910710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cf9be-3d3a-4785-94d2-d92516182de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:49.913291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:49.930758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f46cf9be-3d3a-4785-94d2-d92516182de9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:49.932347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f46cf9be-3d3a-4785-94d2-d92516182de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:49.981846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbd2da-437c-4f99-9331-df828238a24f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:50.018973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 473/941] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:50.021946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbfbc8-b784-420f-b6bb-b66b59911b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:50.022878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbfbc8-b784-420f-b6bb-b66b59911b30 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:50.023163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbfbc8-b784-420f-b6bb-b66b59911b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:50.023443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbfbc8-b784-420f-b6bb-b66b59911b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:50.035221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cf9be-3d3a-4785-94d2-d92516182de9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:50.035856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbfbc8-b784-420f-b6bb-b66b59911b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:50.037337 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbfbc8-b784-420f-b6bb-b66b59911b30 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:50.051998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-29cbfbc8-b784-420f-b6bb-b66b59911b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:50.062837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:50.063790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:50.064716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:50.064716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:50.064716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:50.064716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:50.064716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 469/942] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:50.071379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbfbc8-b784-420f-b6bb-b66b59911b30 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:50.170141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cf9be-3d3a-4785-94d2-d92516182de9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:50.189659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:50.198481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbfbc8-b784-420f-b6bb-b66b59911b30 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:50.213238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:50.214221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:50.214743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:50.215303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:50.216327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:50.217988 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:50.219787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WARNING keystone.server.flask.common [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 20:51:50.233454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06205432-f681-4d0f-ba40-c72bdcf5cd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:50.234166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06205432-f681-4d0f-ba40-c72bdcf5cd96 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:50.234501 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06205432-f681-4d0f-ba40-c72bdcf5cd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:50.234754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06205432-f681-4d0f-ba40-c72bdcf5cd96 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:50.235477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06205432-f681-4d0f-ba40-c72bdcf5cd96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:50.235784 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06205432-f681-4d0f-ba40-c72bdcf5cd96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:50.237218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06205432-f681-4d0f-ba40-c72bdcf5cd96 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:50.242622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.sql.core [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 20:51:50.242622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 20:51:50.242622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [parameters: {'id': '435b75774a5d4d9c9f91b4ada8e4ac53', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 20:51:50.242622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80268) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 20:51:50.254477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 470/943] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:50.260885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06205432-f681-4d0f-ba40-c72bdcf5cd96 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 435b75774a5d4d9c9f91b4ada8e4ac53.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 435b75774a5d4d9c9f91b4ada8e4ac53. May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application conn.query(q) May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:51:50.265598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result.read() May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 26 20:51:50.266622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return project_ref.to_dict() May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application next(self.gen) May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application yield resource May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application next(self.gen) May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.session.rollback() May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.force_reraise() May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise self.value May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application session.commit() May 26 20:51:50.267856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._prepare_impl() May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.session.flush() May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._flush(objects) May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application compat.raise_( May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application flush_context.execute() May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 20:51:50.268977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rec.execute(self) May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 20:51:50.270090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application conn.query(q) May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result.read() May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:51:50.271519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [parameters: {'id': '435b75774a5d4d9c9f91b4ada8e4ac53', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:50.273061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:50.274078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:50.275300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:50.275300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 26 20:51:50.275300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 26 20:51:50.275300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:50.275300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:50.275300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 26 20:51:50.275300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.Conflict( May 26 20:51:50.275300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 435b75774a5d4d9c9f91b4ada8e4ac53. May 26 20:51:50.275300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:50.275300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 474/944] 10.222.0.57 () {60 vars in 1140 bytes} [Thu May 26 20:51:50 2022] POST /identity/v3/projects => generated 262 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:50.278245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:50.356311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:50.376338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06205432-f681-4d0f-ba40-c72bdcf5cd96 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:50.402560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7768334-b5e5-47ab-b359-fa3ad0ec209d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:50.420476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a792ed6-12fd-47bd-94f4-1806ac1627ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:50.421499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a792ed6-12fd-47bd-94f4-1806ac1627ee None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:50.421945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a792ed6-12fd-47bd-94f4-1806ac1627ee None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:50.422372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a792ed6-12fd-47bd-94f4-1806ac1627ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:50.429687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a792ed6-12fd-47bd-94f4-1806ac1627ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:50.431460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a792ed6-12fd-47bd-94f4-1806ac1627ee None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:50.444503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf75e7-5122-411d-a8c9-523cecf29bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:50.445450 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf75e7-5122-411d-a8c9-523cecf29bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:50.445835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf75e7-5122-411d-a8c9-523cecf29bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:50.446213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf75e7-5122-411d-a8c9-523cecf29bf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-6a792ed6-12fd-47bd-94f4-1806ac1627ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:50.446866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:50.447900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:50.448996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:50.448996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:50.448996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:50.448996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:50.448996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:50.448996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:50.448996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:50.448996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:50.455758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 471/945] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:50.456666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf75e7-5122-411d-a8c9-523cecf29bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:50.456946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a792ed6-12fd-47bd-94f4-1806ac1627ee None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:50.458669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf75e7-5122-411d-a8c9-523cecf29bf3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8adf75e7-5122-411d-a8c9-523cecf29bf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:50.475939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:50.477073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:50.478173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:50.478173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:50.478173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:50.478173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:50.478173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:50.478173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:50.478173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:50.478173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:50.480017 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 475/946] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:50.481220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:50.485351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf75e7-5122-411d-a8c9-523cecf29bf3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:50.565528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a792ed6-12fd-47bd-94f4-1806ac1627ee None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:50.598949 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf75e7-5122-411d-a8c9-523cecf29bf3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:50.604573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7f132-dd7e-4e9e-8a0d-cb93df4fe0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:50.605344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7f132-dd7e-4e9e-8a0d-cb93df4fe0bd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:50.605731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7f132-dd7e-4e9e-8a0d-cb93df4fe0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:50.606110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7f132-dd7e-4e9e-8a0d-cb93df4fe0bd None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:50.607162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7f132-dd7e-4e9e-8a0d-cb93df4fe0bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:50.607580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7f132-dd7e-4e9e-8a0d-cb93df4fe0bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:50.609136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7f132-dd7e-4e9e-8a0d-cb93df4fe0bd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:50.621364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 472/947] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:50.622888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e8329-1b70-4593-abfb-c791f348dc68 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:50.623281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e8329-1b70-4593-abfb-c791f348dc68 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:50.623662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e8329-1b70-4593-abfb-c791f348dc68 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:50.625143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 473/948] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:50.627297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e8329-1b70-4593-abfb-c791f348dc68 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:50.631837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:50.640976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bcbd36-aa44-461e-a3ed-d17fe4a22792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:50.641640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbd36-aa44-461e-a3ed-d17fe4a22792 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:50.641925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbd36-aa44-461e-a3ed-d17fe4a22792 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:50.642203 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bcbd36-aa44-461e-a3ed-d17fe4a22792 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:50.642990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbd36-aa44-461e-a3ed-d17fe4a22792 None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:50.643298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbd36-aa44-461e-a3ed-d17fe4a22792 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:50.644715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bcbd36-aa44-461e-a3ed-d17fe4a22792 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:50.674872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 476/949] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:50.677663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa13c7-38a9-4f98-80e4-82a555f2bf8e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:50.678181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa13c7-38a9-4f98-80e4-82a555f2bf8e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:50.678712 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa13c7-38a9-4f98-80e4-82a555f2bf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:50.705216 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1dfa13c7-38a9-4f98-80e4-82a555f2bf8e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:50.706999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1dfa13c7-38a9-4f98-80e4-82a555f2bf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:50.726536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:50.764030 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e8329-1b70-4593-abfb-c791f348dc68 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:50.802918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 477/950] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:50.817105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab610d50-048b-41a3-bd2e-a543475ddf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:50.818194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab610d50-048b-41a3-bd2e-a543475ddf95 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:50.818648 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab610d50-048b-41a3-bd2e-a543475ddf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:50.819612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab610d50-048b-41a3-bd2e-a543475ddf95 None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/e67f497e8a8b46e1b665999ba8fda38d/roles/190ab872a79449cdbc3d501f21d56d63` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:50.823297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab610d50-048b-41a3-bd2e-a543475ddf95 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=e67f497e8a8b46e1b665999ba8fda38d, role_id=190ab872a79449cdbc3d501f21d56d63)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:50.824322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa13c7-38a9-4f98-80e4-82a555f2bf8e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:50.825119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab610d50-048b-41a3-bd2e-a543475ddf95 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:50.846553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 474/951] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:51:50 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/e67f497e8a8b46e1b665999ba8fda38d/roles/190ab872a79449cdbc3d501f21d56d63 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:51:50.954745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa13c7-38a9-4f98-80e4-82a555f2bf8e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:50.978400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:51.003002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82241388-884a-4f2b-bfd0-3231507d3bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:51.003860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82241388-884a-4f2b-bfd0-3231507d3bff None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:51.004147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82241388-884a-4f2b-bfd0-3231507d3bff None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:51.004429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82241388-884a-4f2b-bfd0-3231507d3bff None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:51.005170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82241388-884a-4f2b-bfd0-3231507d3bff None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:51.007214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82241388-884a-4f2b-bfd0-3231507d3bff None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-82241388-884a-4f2b-bfd0-3231507d3bff None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:51.021149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:51.023040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:51.023948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 478/952] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:51.029988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab610d50-048b-41a3-bd2e-a543475ddf95 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:51.137951 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab610d50-048b-41a3-bd2e-a543475ddf95 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:51.173200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf349c19-83d7-4040-abbb-0e116a930f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:51.173876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf349c19-83d7-4040-abbb-0e116a930f01 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:51.174162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf349c19-83d7-4040-abbb-0e116a930f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:51.174465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf349c19-83d7-4040-abbb-0e116a930f01 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:51.175183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf349c19-83d7-4040-abbb-0e116a930f01 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:51.175492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf349c19-83d7-4040-abbb-0e116a930f01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:51.176940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf349c19-83d7-4040-abbb-0e116a930f01 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:51.179377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:51.186547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 475/953] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 26 20:51:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:51.187916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2765ae-bf5d-4005-a6a0-e8de31791ad2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:51.188210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2765ae-bf5d-4005-a6a0-e8de31791ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:51.188493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2765ae-bf5d-4005-a6a0-e8de31791ad2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:51.191345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 479/954] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:51.193615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf349c19-83d7-4040-abbb-0e116a930f01 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:51.204215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf1f9f-0449-460a-af54-2328b2298e94 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:51.204511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf1f9f-0449-460a-af54-2328b2298e94 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:51.204863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf1f9f-0449-460a-af54-2328b2298e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:51.225332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-31bf1f9f-0449-460a-af54-2328b2298e94 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:51.227145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-31bf1f9f-0449-460a-af54-2328b2298e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:51.305280 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf349c19-83d7-4040-abbb-0e116a930f01 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:51.313502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 480/955] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:51.331223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf1f9f-0449-460a-af54-2328b2298e94 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:51.341915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a60b3e-d340-45cb-88b7-2f11acbda6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:51.342611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a60b3e-d340-45cb-88b7-2f11acbda6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:51.342899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a60b3e-d340-45cb-88b7-2f11acbda6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:51.343177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a60b3e-d340-45cb-88b7-2f11acbda6e0 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:51.343890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a60b3e-d340-45cb-88b7-2f11acbda6e0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:51.344198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a60b3e-d340-45cb-88b7-2f11acbda6e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:51.345585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a60b3e-d340-45cb-88b7-2f11acbda6e0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:51.351111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 476/956] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 26 20:51:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:51.359291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a60b3e-d340-45cb-88b7-2f11acbda6e0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:51.430592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:51.456714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf1f9f-0449-460a-af54-2328b2298e94 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:51.471618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a60b3e-d340-45cb-88b7-2f11acbda6e0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:51.476590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:51.500242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6f4bd-4dd7-4c7b-ae0b-be2372da2322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:51.501048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6f4bd-4dd7-4c7b-ae0b-be2372da2322 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:51.501330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6f4bd-4dd7-4c7b-ae0b-be2372da2322 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:51.501603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6f4bd-4dd7-4c7b-ae0b-be2372da2322 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:51.502254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6f4bd-4dd7-4c7b-ae0b-be2372da2322 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:51.503725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6f4bd-4dd7-4c7b-ae0b-be2372da2322 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-50f6f4bd-4dd7-4c7b-ae0b-be2372da2322 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:51.516325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:51.517600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:51.518619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 481/957] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:51.522547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12ff0a-c2eb-4e27-a5d7-4278609a876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:51.523700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12ff0a-c2eb-4e27-a5d7-4278609a876b None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:51.523994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12ff0a-c2eb-4e27-a5d7-4278609a876b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:51.524273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12ff0a-c2eb-4e27-a5d7-4278609a876b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:51.525017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12ff0a-c2eb-4e27-a5d7-4278609a876b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:51.526396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12ff0a-c2eb-4e27-a5d7-4278609a876b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:51.531645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6f4bd-4dd7-4c7b-ae0b-be2372da2322 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:51.558544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 477/958] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 26 20:51:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:51:51.653631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6f4bd-4dd7-4c7b-ae0b-be2372da2322 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:51.698509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b51f25-44d2-4eed-8040-f4253cad9b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:51.699577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b51f25-44d2-4eed-8040-f4253cad9b8a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:51.700060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b51f25-44d2-4eed-8040-f4253cad9b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:51.700522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b51f25-44d2-4eed-8040-f4253cad9b8a None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:51.701484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b51f25-44d2-4eed-8040-f4253cad9b8a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:51.702093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b51f25-44d2-4eed-8040-f4253cad9b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:51.703777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b51f25-44d2-4eed-8040-f4253cad9b8a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:51.714527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 482/959] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:51.716679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12ff0a-c2eb-4e27-a5d7-4278609a876b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:51.841472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12ff0a-c2eb-4e27-a5d7-4278609a876b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:51.863272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:51.878983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716f1e0-29c4-49ff-a071-d881fa5d6f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:51.879650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716f1e0-29c4-49ff-a071-d881fa5d6f50 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:51.879935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716f1e0-29c4-49ff-a071-d881fa5d6f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:51.880213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716f1e0-29c4-49ff-a071-d881fa5d6f50 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:51.880928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716f1e0-29c4-49ff-a071-d881fa5d6f50 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:51.881236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716f1e0-29c4-49ff-a071-d881fa5d6f50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:51.882664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716f1e0-29c4-49ff-a071-d881fa5d6f50 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:51.888085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 478/960] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:53.008883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fffde5-b6d1-4e7b-8dcc-7b4d874b5f49 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.009440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fffde5-b6d1-4e7b-8dcc-7b4d874b5f49 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.009440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fffde5-b6d1-4e7b-8dcc-7b4d874b5f49 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.011402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 483/961] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:53.017353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cd2cf-4a65-4316-a4c0-b0f2dcfeba0b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.017853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cd2cf-4a65-4316-a4c0-b0f2dcfeba0b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.017964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cd2cf-4a65-4316-a4c0-b0f2dcfeba0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.035254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7b1cd2cf-4a65-4316-a4c0-b0f2dcfeba0b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:53.036430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7b1cd2cf-4a65-4316-a4c0-b0f2dcfeba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:53.107803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 479/962] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:53.132435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fffde5-b6d1-4e7b-8dcc-7b4d874b5f49 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:53.142209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2887-637f-41dc-8007-5fd5157b4971 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.142528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2887-637f-41dc-8007-5fd5157b4971 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.142810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2887-637f-41dc-8007-5fd5157b4971 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.144406 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 480/963] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:53.149269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81db5f-0e8c-4476-9183-ef086e9eacb0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.149554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81db5f-0e8c-4476-9183-ef086e9eacb0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.149835 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81db5f-0e8c-4476-9183-ef086e9eacb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.167791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4e81db5f-0e8c-4476-9183-ef086e9eacb0 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:53.168957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4e81db5f-0e8c-4476-9183-ef086e9eacb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:53.171630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:53.238179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fffde5-b6d1-4e7b-8dcc-7b4d874b5f49 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:53.240743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 481/964] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:53.265853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81db5f-0e8c-4476-9183-ef086e9eacb0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:53.272775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa9d63-66ca-4bb3-a86b-8b235e9699c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:53.273534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa9d63-66ca-4bb3-a86b-8b235e9699c1 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.273818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa9d63-66ca-4bb3-a86b-8b235e9699c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.274222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa9d63-66ca-4bb3-a86b-8b235e9699c1 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.275001 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa9d63-66ca-4bb3-a86b-8b235e9699c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:53.276564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa9d63-66ca-4bb3-a86b-8b235e9699c1 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:53.292235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.sql.core [None req-73fa9d63-66ca-4bb3-a86b-8b235e9699c1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 20:51:53.292235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 20:51:53.292235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [parameters: {'user_id': 'ac165d508cb848debb3d606373aa60be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 20:51:53.292235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80268) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-73fa9d63-66ca-4bb3-a86b-8b235e9699c1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application conn.query(q) May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result.read() May 26 20:51:53.303877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:51:53.304990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application next(self.gen) May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application yield resource May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application next(self.gen) May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.session.rollback() May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.force_reraise() May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise self.value May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application session.commit() May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 20:51:53.306044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._prepare_impl() May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.session.flush() May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._flush(objects) May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application compat.raise_( May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application flush_context.execute() May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rec.execute(self) May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 20:51:53.307408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result = connection._execute_20( May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 20:51:53.309057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result = self._query(query) May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application conn.query(q) May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application result.read() May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 20:51:53.310606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ac165d508cb848debb3d606373aa60be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:53.315468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 26 20:51:53.316287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 26 20:51:53.317186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 26 20:51:53.317186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 26 20:51:53.317186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 20:51:53.317186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 20:51:53.317186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 20:51:53.317186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:53.317186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 484/965] 10.222.0.57 () {60 vars in 1131 bytes} [Thu May 26 20:51:53 2022] POST /identity/v3/users => generated 160 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:53.317186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa9d63-66ca-4bb3-a86b-8b235e9699c1 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:53.320410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:53.391001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81db5f-0e8c-4476-9183-ef086e9eacb0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:53.428591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:53.433139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa9d63-66ca-4bb3-a86b-8b235e9699c1 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:53.437732 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fce94c-0bb2-4956-b42e-b95705ca8fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:53.438783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fce94c-0bb2-4956-b42e-b95705ca8fbb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.439167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fce94c-0bb2-4956-b42e-b95705ca8fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.439543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fce94c-0bb2-4956-b42e-b95705ca8fbb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.453757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fce94c-0bb2-4956-b42e-b95705ca8fbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a6fce94c-0bb2-4956-b42e-b95705ca8fbb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:51:53.465410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:53.466601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:51:53.467537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:53.470589 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 482/966] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 26 20:51:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:53.473264 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fce94c-0bb2-4956-b42e-b95705ca8fbb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:53.486317 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6bd9e-ac01-4a69-b00b-474703e37c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:53.487282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6bd9e-ac01-4a69-b00b-474703e37c81 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.487678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6bd9e-ac01-4a69-b00b-474703e37c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.488113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6bd9e-ac01-4a69-b00b-474703e37c81 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.499360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6bd9e-ac01-4a69-b00b-474703e37c81 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:53.502017 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6bd9e-ac01-4a69-b00b-474703e37c81 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1be6bd9e-ac01-4a69-b00b-474703e37c81 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:53.521065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:53.522625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:53.523804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:53.523804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:53.523804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:53.523804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:53.523804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:53.523804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:53.523804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:53.523804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:53.523804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 20:51:53.523804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:53.531121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 485/967] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:51:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:53.532190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6bd9e-ac01-4a69-b00b-474703e37c81 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:53.604892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fce94c-0bb2-4956-b42e-b95705ca8fbb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:53.607911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:53.644098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5a0db-ea70-4ed1-a8fc-effd1793bbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:53.645041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5a0db-ea70-4ed1-a8fc-effd1793bbde None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.645558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5a0db-ea70-4ed1-a8fc-effd1793bbde None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.646012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5a0db-ea70-4ed1-a8fc-effd1793bbde None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.647550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5a0db-ea70-4ed1-a8fc-effd1793bbde None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:53.648034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5a0db-ea70-4ed1-a8fc-effd1793bbde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:53.649101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6bd9e-ac01-4a69-b00b-474703e37c81 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:53.649819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5a0db-ea70-4ed1-a8fc-effd1793bbde None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:53.659176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 483/968] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 26 20:51:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:53.665330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5a0db-ea70-4ed1-a8fc-effd1793bbde None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:53.697015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aece70-ef50-46e3-8070-68bc9f9027cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:53.697802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aece70-ef50-46e3-8070-68bc9f9027cf None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.698160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aece70-ef50-46e3-8070-68bc9f9027cf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.698760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aece70-ef50-46e3-8070-68bc9f9027cf None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.699636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aece70-ef50-46e3-8070-68bc9f9027cf None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:53.700055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aece70-ef50-46e3-8070-68bc9f9027cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:53.702190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aece70-ef50-46e3-8070-68bc9f9027cf None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:53.758758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 486/969] 10.222.0.57 () {56 vars in 1101 bytes} [Thu May 26 20:51:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:53.792518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5a0db-ea70-4ed1-a8fc-effd1793bbde None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:53.833145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b4045-e436-49ec-bac1-eba3f7b6efc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:53.833962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b4045-e436-49ec-bac1-eba3f7b6efc4 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.834246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b4045-e436-49ec-bac1-eba3f7b6efc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.834641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b4045-e436-49ec-bac1-eba3f7b6efc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.848476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b4045-e436-49ec-bac1-eba3f7b6efc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:53.850006 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b4045-e436-49ec-bac1-eba3f7b6efc4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-cb6b4045-e436-49ec-bac1-eba3f7b6efc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:53.868967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:53.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:53.870888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:53.870888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:53.870888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:53.870888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:53.870888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 484/970] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:53.876343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aece70-ef50-46e3-8070-68bc9f9027cf None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:53.897866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:53.961880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef93fd0-66d0-4c69-943c-42e3976d597f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.962195 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef93fd0-66d0-4c69-943c-42e3976d597f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.962436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef93fd0-66d0-4c69-943c-42e3976d597f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.964307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 485/971] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:53.969217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45628e83-299c-48da-b962-859dae2124e8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:53.969499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45628e83-299c-48da-b962-859dae2124e8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:53.969774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45628e83-299c-48da-b962-859dae2124e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:53.989095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-45628e83-299c-48da-b962-859dae2124e8 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:53.990279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-45628e83-299c-48da-b962-859dae2124e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:53.990872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aece70-ef50-46e3-8070-68bc9f9027cf None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:54.005649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:54.041347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8f9ac-a70b-4cdf-a5a0-8db3c78a8544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:54.042040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8f9ac-a70b-4cdf-a5a0-8db3c78a8544 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.042324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8f9ac-a70b-4cdf-a5a0-8db3c78a8544 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.042640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8f9ac-a70b-4cdf-a5a0-8db3c78a8544 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.043382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8f9ac-a70b-4cdf-a5a0-8db3c78a8544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:54.043694 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8f9ac-a70b-4cdf-a5a0-8db3c78a8544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:54.045105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8f9ac-a70b-4cdf-a5a0-8db3c78a8544 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:54.060174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 487/972] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:54.065842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8f9ac-a70b-4cdf-a5a0-8db3c78a8544 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:54.085929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 486/973] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:54.119636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45628e83-299c-48da-b962-859dae2124e8 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:54.164557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:54.189179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8f9ac-a70b-4cdf-a5a0-8db3c78a8544 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:54.224875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c26abc-e1fb-403d-9934-88d03598e2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:54.225389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45628e83-299c-48da-b962-859dae2124e8 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:54.225718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c26abc-e1fb-403d-9934-88d03598e2ba None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.226002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c26abc-e1fb-403d-9934-88d03598e2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.226279 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c26abc-e1fb-403d-9934-88d03598e2ba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.233824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c26abc-e1fb-403d-9934-88d03598e2ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:54.235336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c26abc-e1fb-403d-9934-88d03598e2ba None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-44c26abc-e1fb-403d-9934-88d03598e2ba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:51:54.250091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:54.251089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:54.252044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 488/974] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 26 20:51:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:54.252662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0df6-84c8-4e99-b257-f95d26132626 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.252866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0df6-84c8-4e99-b257-f95d26132626 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.253075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb0df6-84c8-4e99-b257-f95d26132626 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.254529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 489/975] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:54.257723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0df6-84c8-4e99-b257-f95d26132626 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:54.263083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676bb47-7dca-4d16-bba6-b97f1f9c5506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:54.263888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676bb47-7dca-4d16-bba6-b97f1f9c5506 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.264172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676bb47-7dca-4d16-bba6-b97f1f9c5506 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.264449 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676bb47-7dca-4d16-bba6-b97f1f9c5506 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.265109 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676bb47-7dca-4d16-bba6-b97f1f9c5506 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:54.266506 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676bb47-7dca-4d16-bba6-b97f1f9c5506 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-7676bb47-7dca-4d16-bba6-b97f1f9c5506 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:54.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:54.279572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:54.280429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 487/976] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:54.281830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21016b2b-ce26-4ac4-9325-46f93a9b245f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.281830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21016b2b-ce26-4ac4-9325-46f93a9b245f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.281830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21016b2b-ce26-4ac4-9325-46f93a9b245f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.299076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-21016b2b-ce26-4ac4-9325-46f93a9b245f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:54.300267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-21016b2b-ce26-4ac4-9325-46f93a9b245f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:54.308125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:54.360424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb0df6-84c8-4e99-b257-f95d26132626 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:54.367886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 488/977] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:54.368698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21016b2b-ce26-4ac4-9325-46f93a9b245f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:54.396785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b2535-a11b-4597-92ff-a09b906819eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:54.397732 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b2535-a11b-4597-92ff-a09b906819eb None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.398158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b2535-a11b-4597-92ff-a09b906819eb None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.398493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b2535-a11b-4597-92ff-a09b906819eb None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.399330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b2535-a11b-4597-92ff-a09b906819eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:54.399642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b2535-a11b-4597-92ff-a09b906819eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:54.401063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b2535-a11b-4597-92ff-a09b906819eb None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:54.403612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:54.421170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 490/978] 10.222.0.57 () {56 vars in 1149 bytes} [Thu May 26 20:51:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:54.421904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b2535-a11b-4597-92ff-a09b906819eb None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:54.469514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21016b2b-ce26-4ac4-9325-46f93a9b245f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:54.502836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa76af3-1f60-49f7-baea-9b7111624f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:54.503488 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa76af3-1f60-49f7-baea-9b7111624f50 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.503771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa76af3-1f60-49f7-baea-9b7111624f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.504058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa76af3-1f60-49f7-baea-9b7111624f50 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.504754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa76af3-1f60-49f7-baea-9b7111624f50 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:54.505249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa76af3-1f60-49f7-baea-9b7111624f50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:54.506478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa76af3-1f60-49f7-baea-9b7111624f50 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:54.511336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 489/979] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 26 20:51:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:54.512101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa76af3-1f60-49f7-baea-9b7111624f50 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:54.520006 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b2535-a11b-4597-92ff-a09b906819eb None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:54.523550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:54.553053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a2b8d-6309-448d-8872-082728ace20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:54.554259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a2b8d-6309-448d-8872-082728ace20c None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.554574 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a2b8d-6309-448d-8872-082728ace20c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.554856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a2b8d-6309-448d-8872-082728ace20c None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.555565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a2b8d-6309-448d-8872-082728ace20c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:54.557105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a2b8d-6309-448d-8872-082728ace20c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:54.582473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 491/980] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:51:54 2022] POST /identity/v3/services => generated 242 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:51:54.582473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a2b8d-6309-448d-8872-082728ace20c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:54.627521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa76af3-1f60-49f7-baea-9b7111624f50 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:54.633425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:54.672936 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89050eb6-0a61-469d-ad08-42cfd308ce7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:54.673504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050eb6-0a61-469d-ad08-42cfd308ce7c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.673789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050eb6-0a61-469d-ad08-42cfd308ce7c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.674065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89050eb6-0a61-469d-ad08-42cfd308ce7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.691191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a2b8d-6309-448d-8872-082728ace20c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:54.729163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168919d-2d25-478c-ab3f-23f1148234c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:54.729991 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168919d-2d25-478c-ab3f-23f1148234c2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.730276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168919d-2d25-478c-ab3f-23f1148234c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.730585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168919d-2d25-478c-ab3f-23f1148234c2 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.731306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168919d-2d25-478c-ab3f-23f1148234c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:54.731619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168919d-2d25-478c-ab3f-23f1148234c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:54.733259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168919d-2d25-478c-ab3f-23f1148234c2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:54.739424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 492/981] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 26 20:51:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:54.745821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168919d-2d25-478c-ab3f-23f1148234c2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:54.747115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89050eb6-0a61-469d-ad08-42cfd308ce7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:54.748630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89050eb6-0a61-469d-ad08-42cfd308ce7c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:54.750607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:54.856252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168919d-2d25-478c-ab3f-23f1148234c2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:54.869835 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 490/982] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:51:54 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:51:54.870987 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89050eb6-0a61-469d-ad08-42cfd308ce7c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:54.877906 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:54.898131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f988e-03ad-4817-8839-56cb452cdf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:54.898833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f988e-03ad-4817-8839-56cb452cdf4a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:54.899171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f988e-03ad-4817-8839-56cb452cdf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:54.899452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f988e-03ad-4817-8839-56cb452cdf4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:54.900281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f988e-03ad-4817-8839-56cb452cdf4a None admin] RBAC: Adding query filter params (interface=admin, service_id=3e14356ca9114885ab9ec88c8831dbdd, region_id=RegionOne) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:54.900627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f988e-03ad-4817-8839-56cb452cdf4a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:54.901999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f988e-03ad-4817-8839-56cb452cdf4a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:54.907462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 493/983] 10.222.0.57 () {56 vars in 1249 bytes} [Thu May 26 20:51:54 2022] GET /identity/v3/endpoints?service_id=3e14356ca9114885ab9ec88c8831dbdd&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:54.913127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f988e-03ad-4817-8839-56cb452cdf4a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:54.968959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89050eb6-0a61-469d-ad08-42cfd308ce7c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:54.970416 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:55.005830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010778b-ffd2-4489-9546-cfb01075bb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:55.006692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010778b-ffd2-4489-9546-cfb01075bb61 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:55.006977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010778b-ffd2-4489-9546-cfb01075bb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:55.007242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010778b-ffd2-4489-9546-cfb01075bb61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:55.020179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010778b-ffd2-4489-9546-cfb01075bb61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:55.021651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010778b-ffd2-4489-9546-cfb01075bb61 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:55.023669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f988e-03ad-4817-8839-56cb452cdf4a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c010778b-ffd2-4489-9546-cfb01075bb61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:51:55.040894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:55.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:55.042810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:51:55.042810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:51:55.042810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:51:55.042810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:55.042810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 491/984] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:55.047611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010778b-ffd2-4489-9546-cfb01075bb61 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:55.062241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6043862-83e4-4968-976f-b71647b50c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:55.063037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6043862-83e4-4968-976f-b71647b50c3a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:55.063328 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6043862-83e4-4968-976f-b71647b50c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:55.063614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6043862-83e4-4968-976f-b71647b50c3a None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:55.064285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6043862-83e4-4968-976f-b71647b50c3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:55.065638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6043862-83e4-4968-976f-b71647b50c3a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:55.072041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 494/985] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 26 20:51:54 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:51:55.166918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010778b-ffd2-4489-9546-cfb01075bb61 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:55.203464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a0039-b69a-4e04-8af2-aeef322dcb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:55.204139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a0039-b69a-4e04-8af2-aeef322dcb51 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:55.204421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a0039-b69a-4e04-8af2-aeef322dcb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:55.204696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a0039-b69a-4e04-8af2-aeef322dcb51 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:55.205422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a0039-b69a-4e04-8af2-aeef322dcb51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:55.205727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a0039-b69a-4e04-8af2-aeef322dcb51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:55.207166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a0039-b69a-4e04-8af2-aeef322dcb51 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:55.223987 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 492/986] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:55.226135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6043862-83e4-4968-976f-b71647b50c3a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:55.277495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:55.332041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6043862-83e4-4968-976f-b71647b50c3a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:55.366657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0557c-0a0d-4699-b3fa-ce4c7f537f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:55.367453 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0557c-0a0d-4699-b3fa-ce4c7f537f99 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:55.367735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0557c-0a0d-4699-b3fa-ce4c7f537f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:55.368010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0557c-0a0d-4699-b3fa-ce4c7f537f99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:55.374862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0557c-0a0d-4699-b3fa-ce4c7f537f99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:55.376299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0557c-0a0d-4699-b3fa-ce4c7f537f99 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-16a0557c-0a0d-4699-b3fa-ce4c7f537f99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:51:55.390651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:55.391569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:55.392484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:51:55.392484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:51:55.392484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:51:55.392484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:51:55.392484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:51:55.392484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:51:55.392484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:51:55.392484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:55.392484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 495/987] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:55.398838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a0039-b69a-4e04-8af2-aeef322dcb51 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:55.416194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:55.501319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a0039-b69a-4e04-8af2-aeef322dcb51 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:55.533371 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d828c-8d98-4b82-a053-8955b2a96175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:55.534054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d828c-8d98-4b82-a053-8955b2a96175 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:55.534338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d828c-8d98-4b82-a053-8955b2a96175 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:55.534642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d828c-8d98-4b82-a053-8955b2a96175 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:55.535422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d828c-8d98-4b82-a053-8955b2a96175 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:55.535732 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d828c-8d98-4b82-a053-8955b2a96175 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:55.537165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d828c-8d98-4b82-a053-8955b2a96175 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:55.549627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 493/988] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:55.555882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0557c-0a0d-4699-b3fa-ce4c7f537f99 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:55.656444 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0557c-0a0d-4699-b3fa-ce4c7f537f99 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:55.664254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:55.691575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45d0e2-2c14-4c5c-ba5b-91dd3aabea91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:55.692224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45d0e2-2c14-4c5c-ba5b-91dd3aabea91 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:55.692503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45d0e2-2c14-4c5c-ba5b-91dd3aabea91 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:55.692777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45d0e2-2c14-4c5c-ba5b-91dd3aabea91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:55.693615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45d0e2-2c14-4c5c-ba5b-91dd3aabea91 None admin] RBAC: Adding query filter params (role.id=190ab872a79449cdbc3d501f21d56d63, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=e67f497e8a8b46e1b665999ba8fda38d) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:55.693954 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45d0e2-2c14-4c5c-ba5b-91dd3aabea91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:55.695398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45d0e2-2c14-4c5c-ba5b-91dd3aabea91 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:55.706369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 496/989] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:51:55 2022] GET /identity/v3/role_assignments?user.id=e67f497e8a8b46e1b665999ba8fda38d&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=190ab872a79449cdbc3d501f21d56d63 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:56.798819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbccfdc-bb6a-48e4-9b25-e584e245dea6 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:56.799387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbccfdc-bb6a-48e4-9b25-e584e245dea6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:56.799387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbccfdc-bb6a-48e4-9b25-e584e245dea6 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:56.801996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 494/990] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:56.806550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479365c-30bb-4158-92ee-221837a3ec4e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:56.807055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479365c-30bb-4158-92ee-221837a3ec4e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:56.807134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479365c-30bb-4158-92ee-221837a3ec4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:56.825434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2479365c-30bb-4158-92ee-221837a3ec4e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:56.826984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2479365c-30bb-4158-92ee-221837a3ec4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:56.893411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 497/991] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:51:56.915993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbccfdc-bb6a-48e4-9b25-e584e245dea6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:56.990053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:57.016765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbccfdc-bb6a-48e4-9b25-e584e245dea6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:57.049721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3896dc-c4cb-4c8e-b00f-0230427c70e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:57.050460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3896dc-c4cb-4c8e-b00f-0230427c70e9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.050747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3896dc-c4cb-4c8e-b00f-0230427c70e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.051023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3896dc-c4cb-4c8e-b00f-0230427c70e9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.051835 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3896dc-c4cb-4c8e-b00f-0230427c70e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:57.053733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3896dc-c4cb-4c8e-b00f-0230427c70e9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0a3896dc-c4cb-4c8e-b00f-0230427c70e9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:57.065313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:57.066316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:51:57.067474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:57.070788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 495/992] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:51:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:51:57.072937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479365c-30bb-4158-92ee-221837a3ec4e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:57.171327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:57.175049 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479365c-30bb-4158-92ee-221837a3ec4e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:57.209031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f6c537-788e-4408-88b8-671e6dd0914b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:57.209720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6c537-788e-4408-88b8-671e6dd0914b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.210000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6c537-788e-4408-88b8-671e6dd0914b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.210275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f6c537-788e-4408-88b8-671e6dd0914b None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.211000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6c537-788e-4408-88b8-671e6dd0914b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:57.211309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6c537-788e-4408-88b8-671e6dd0914b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:57.212673 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f6c537-788e-4408-88b8-671e6dd0914b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:57.217571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 498/993] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:57.222994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3896dc-c4cb-4c8e-b00f-0230427c70e9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:57.278230 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b87e6-0529-4fff-80ee-956bd445e756 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.278536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b87e6-0529-4fff-80ee-956bd445e756 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.278819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b87e6-0529-4fff-80ee-956bd445e756 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.280307 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 499/994] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:57.285904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415414a-9c16-484d-afb9-0ea626829972 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.286194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415414a-9c16-484d-afb9-0ea626829972 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.286502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415414a-9c16-484d-afb9-0ea626829972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.306789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1415414a-9c16-484d-afb9-0ea626829972 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:57.308369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1415414a-9c16-484d-afb9-0ea626829972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:57.325239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3896dc-c4cb-4c8e-b00f-0230427c70e9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:57.358810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb9ad8-7afc-40ed-bc97-73d0244f35d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:57.359642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb9ad8-7afc-40ed-bc97-73d0244f35d8 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.359927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb9ad8-7afc-40ed-bc97-73d0244f35d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.360267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb9ad8-7afc-40ed-bc97-73d0244f35d8 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.360975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb9ad8-7afc-40ed-bc97-73d0244f35d8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:57.361369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb9ad8-7afc-40ed-bc97-73d0244f35d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:57.362730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb9ad8-7afc-40ed-bc97-73d0244f35d8 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:57.368177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 496/995] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:51:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:57.373766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb9ad8-7afc-40ed-bc97-73d0244f35d8 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:57.376766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:57.387761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 500/996] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:57.410408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415414a-9c16-484d-afb9-0ea626829972 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:57.485131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb9ad8-7afc-40ed-bc97-73d0244f35d8 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:57.508208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:57.517230 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415414a-9c16-484d-afb9-0ea626829972 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:57.541526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a562a6-de77-453b-87c1-54e6b77ab03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:57.541526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a562a6-de77-453b-87c1-54e6b77ab03c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.541526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a562a6-de77-453b-87c1-54e6b77ab03c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.541860 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a562a6-de77-453b-87c1-54e6b77ab03c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.543180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a562a6-de77-453b-87c1-54e6b77ab03c None admin] RBAC: Adding query filter params (interface=public, service_id=93c186e5090e463282174bd496896d86, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:57.543496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a562a6-de77-453b-87c1-54e6b77ab03c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:57.545061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a562a6-de77-453b-87c1-54e6b77ab03c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:57.550995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 497/997] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 26 20:51:57 2022] GET /identity/v3/endpoints?service_id=93c186e5090e463282174bd496896d86&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:57.556241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a562a6-de77-453b-87c1-54e6b77ab03c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:57.562342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c78cf-bd9c-44df-92d7-745cf08e4fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:57.563126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c78cf-bd9c-44df-92d7-745cf08e4fea None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.563415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c78cf-bd9c-44df-92d7-745cf08e4fea None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.563691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c78cf-bd9c-44df-92d7-745cf08e4fea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.564349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c78cf-bd9c-44df-92d7-745cf08e4fea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:57.565729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c78cf-bd9c-44df-92d7-745cf08e4fea None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-429c78cf-bd9c-44df-92d7-745cf08e4fea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:57.578698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:57.579735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:51:57.580672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 501/998] 10.222.0.57 () {56 vars in 1126 bytes} [Thu May 26 20:51:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:57.586395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c78cf-bd9c-44df-92d7-745cf08e4fea None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:57.629998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:57.671394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a562a6-de77-453b-87c1-54e6b77ab03c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:57.700217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c78cf-bd9c-44df-92d7-745cf08e4fea None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:57.707929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb5ee8-9f17-43a1-ad04-25c8c593c289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:57.708594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb5ee8-9f17-43a1-ad04-25c8c593c289 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.708879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb5ee8-9f17-43a1-ad04-25c8c593c289 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.709158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb5ee8-9f17-43a1-ad04-25c8c593c289 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.710386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb5ee8-9f17-43a1-ad04-25c8c593c289 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:57.711997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb5ee8-9f17-43a1-ad04-25c8c593c289 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:57.718257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 498/999] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 26 20:51:57 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 20:51:57.737723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68658a-e192-4b1c-bbb2-07e1c7a230cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:57.738468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68658a-e192-4b1c-bbb2-07e1c7a230cc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.738762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68658a-e192-4b1c-bbb2-07e1c7a230cc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.739049 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68658a-e192-4b1c-bbb2-07e1c7a230cc None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.739813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68658a-e192-4b1c-bbb2-07e1c7a230cc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:57.740134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68658a-e192-4b1c-bbb2-07e1c7a230cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:57.741627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68658a-e192-4b1c-bbb2-07e1c7a230cc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:57.747807 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 502/1000] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 26 20:51:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:51:57.753657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb5ee8-9f17-43a1-ad04-25c8c593c289 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:57.876542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb5ee8-9f17-43a1-ad04-25c8c593c289 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:57.908408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:57.916988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d30ab5-73a5-4d61-8bf6-9b6d04cd0bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:57.917705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d30ab5-73a5-4d61-8bf6-9b6d04cd0bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.917991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d30ab5-73a5-4d61-8bf6-9b6d04cd0bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.918318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d30ab5-73a5-4d61-8bf6-9b6d04cd0bd9 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.919455 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d30ab5-73a5-4d61-8bf6-9b6d04cd0bd9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:57.919455 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d30ab5-73a5-4d61-8bf6-9b6d04cd0bd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:57.920807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d30ab5-73a5-4d61-8bf6-9b6d04cd0bd9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:57.925967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 499/1001] 10.222.0.57 () {56 vars in 1122 bytes} [Thu May 26 20:51:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:57.932395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68658a-e192-4b1c-bbb2-07e1c7a230cc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:57.939229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c810a2-339f-4437-87e9-59ae52a495cd None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.939521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c810a2-339f-4437-87e9-59ae52a495cd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.939827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c810a2-339f-4437-87e9-59ae52a495cd None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.941388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 500/1002] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:51:57.947231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d1abe-0861-4517-a22c-daa14d825e4d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:57.947624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d1abe-0861-4517-a22c-daa14d825e4d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:57.947904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d1abe-0861-4517-a22c-daa14d825e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:57.960651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:57.964908 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2d9d1abe-0861-4517-a22c-daa14d825e4d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:57.966018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2d9d1abe-0861-4517-a22c-daa14d825e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:51:58.038076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 501/1003] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:51:58.042174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68658a-e192-4b1c-bbb2-07e1c7a230cc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:58.060116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d1abe-0861-4517-a22c-daa14d825e4d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:58.079003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22c894-0461-4a0f-9298-01fb53c2bb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:58.079598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22c894-0461-4a0f-9298-01fb53c2bb7d None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:58.079886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22c894-0461-4a0f-9298-01fb53c2bb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:58.080166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22c894-0461-4a0f-9298-01fb53c2bb7d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:58.081068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22c894-0461-4a0f-9298-01fb53c2bb7d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:58.082920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22c894-0461-4a0f-9298-01fb53c2bb7d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:58.099468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 503/1004] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 26 20:51:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:51:58.179352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d1abe-0861-4517-a22c-daa14d825e4d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:58.206090 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:58.225799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a41f1-6333-456d-81e6-32459c5c362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:58.226643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a41f1-6333-456d-81e6-32459c5c362b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:58.226928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a41f1-6333-456d-81e6-32459c5c362b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:58.227204 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a41f1-6333-456d-81e6-32459c5c362b None admin] PATH_INFO: `/v3/services/image` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:58.227869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a41f1-6333-456d-81e6-32459c5c362b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:58.229290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a41f1-6333-456d-81e6-32459c5c362b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a64a41f1-6333-456d-81e6-32459c5c362b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:51:58.242701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:51:58.243797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:51:58.244669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 502/1005] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 26 20:51:58 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:51:58.254514 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22c894-0461-4a0f-9298-01fb53c2bb7d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:58.339007 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:51:58.360472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22c894-0461-4a0f-9298-01fb53c2bb7d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:58.395332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607fc6b3-c444-4f01-857c-0752352cc0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:58.396003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fc6b3-c444-4f01-857c-0752352cc0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:58.396294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fc6b3-c444-4f01-857c-0752352cc0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:58.396575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fc6b3-c444-4f01-857c-0752352cc0f7 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:58.397325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fc6b3-c444-4f01-857c-0752352cc0f7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:58.397643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fc6b3-c444-4f01-857c-0752352cc0f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:58.399049 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607fc6b3-c444-4f01-857c-0752352cc0f7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:58.404231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 504/1006] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:58.409961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a41f1-6333-456d-81e6-32459c5c362b None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:51:58.512668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a41f1-6333-456d-81e6-32459c5c362b None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:51:58.546137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f24a39-c59d-46d7-9c00-3252e1315079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:51:58.546834 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f24a39-c59d-46d7-9c00-3252e1315079 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:58.547121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f24a39-c59d-46d7-9c00-3252e1315079 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:58.547401 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f24a39-c59d-46d7-9c00-3252e1315079 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:58.548109 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f24a39-c59d-46d7-9c00-3252e1315079 None admin] RBAC: Adding query filter params (type=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:51:58.548421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f24a39-c59d-46d7-9c00-3252e1315079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:51:58.549795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f24a39-c59d-46d7-9c00-3252e1315079 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:51:58.558520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 503/1007] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:51:58 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:51:59.934750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f8945-abc1-4518-a07a-140923cda3d5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:59.935260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f8945-abc1-4518-a07a-140923cda3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:59.935338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f8945-abc1-4518-a07a-140923cda3d5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:59.936836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 505/1008] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:51:59.941787 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ebc2e-d7d1-48f8-bf0b-e0ec546333d9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:51:59.942170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ebc2e-d7d1-48f8-bf0b-e0ec546333d9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:51:59.942355 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ebc2e-d7d1-48f8-bf0b-e0ec546333d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:51:59.959952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8b0ebc2e-d7d1-48f8-bf0b-e0ec546333d9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:51:59.961133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8b0ebc2e-d7d1-48f8-bf0b-e0ec546333d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:00.005101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:00.034650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 504/1009] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:51:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:00.057336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f8945-abc1-4518-a07a-140923cda3d5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:00.164078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f8945-abc1-4518-a07a-140923cda3d5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:00.200046 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57b556-8a94-4faf-b9d2-3b04bf4e8085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:00.200844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b556-8a94-4faf-b9d2-3b04bf4e8085 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:00.201127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b556-8a94-4faf-b9d2-3b04bf4e8085 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:00.201404 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b556-8a94-4faf-b9d2-3b04bf4e8085 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:00.202054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57b556-8a94-4faf-b9d2-3b04bf4e8085 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:00.203465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57b556-8a94-4faf-b9d2-3b04bf4e8085 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-6d57b556-8a94-4faf-b9d2-3b04bf4e8085 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:00.215790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:00.216805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 20:52:00.217680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:00.219125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 506/1010] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:52:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:00.223141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ebc2e-d7d1-48f8-bf0b-e0ec546333d9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:00.321260 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ebc2e-d7d1-48f8-bf0b-e0ec546333d9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:00.339846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fcbce-3780-405d-9b96-f2f897ea24fc None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:00.340139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fcbce-3780-405d-9b96-f2f897ea24fc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:00.340424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fcbce-3780-405d-9b96-f2f897ea24fc None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:00.341925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 507/1011] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:00.346824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983639f6-9c34-48ff-a8e8-f3ad1185a19e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:00.347118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983639f6-9c34-48ff-a8e8-f3ad1185a19e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:00.347402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983639f6-9c34-48ff-a8e8-f3ad1185a19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:00.364591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca7b4b-f11f-47fb-9b04-5f0369cab2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:00.364796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-983639f6-9c34-48ff-a8e8-f3ad1185a19e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:00.365651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca7b4b-f11f-47fb-9b04-5f0369cab2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:00.366017 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-983639f6-9c34-48ff-a8e8-f3ad1185a19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:00.366239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca7b4b-f11f-47fb-9b04-5f0369cab2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:00.366688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca7b4b-f11f-47fb-9b04-5f0369cab2e3 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:00.367552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca7b4b-f11f-47fb-9b04-5f0369cab2e3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:00.368017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca7b4b-f11f-47fb-9b04-5f0369cab2e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:00.368969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:00.369549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca7b4b-f11f-47fb-9b04-5f0369cab2e3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:00.379330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 505/1012] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:52:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:00.380529 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca7b4b-f11f-47fb-9b04-5f0369cab2e3 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:00.428572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:00.443392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 508/1013] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:00 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:00.467341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983639f6-9c34-48ff-a8e8-f3ad1185a19e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:00.495781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca7b4b-f11f-47fb-9b04-5f0369cab2e3 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:00.533026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a3337-a57f-4c4b-b29e-0f491f4002c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:00.533685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a3337-a57f-4c4b-b29e-0f491f4002c9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:00.534179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a3337-a57f-4c4b-b29e-0f491f4002c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:00.534484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a3337-a57f-4c4b-b29e-0f491f4002c9 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:00.535351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a3337-a57f-4c4b-b29e-0f491f4002c9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:00.535634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a3337-a57f-4c4b-b29e-0f491f4002c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:00.537021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a3337-a57f-4c4b-b29e-0f491f4002c9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:00.547238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 506/1014] 10.222.0.57 () {56 vars in 1112 bytes} [Thu May 26 20:52:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:00.548242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a3337-a57f-4c4b-b29e-0f491f4002c9 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:00.581436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983639f6-9c34-48ff-a8e8-f3ad1185a19e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:00.619077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f9996-7323-42c2-95a3-8a2bbf72a4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:00.619874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f9996-7323-42c2-95a3-8a2bbf72a4df None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:00.620162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f9996-7323-42c2-95a3-8a2bbf72a4df None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:00.620443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f9996-7323-42c2-95a3-8a2bbf72a4df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:00.621625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f9996-7323-42c2-95a3-8a2bbf72a4df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:00.623105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f9996-7323-42c2-95a3-8a2bbf72a4df None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:00.624893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 509/1015] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:00.630219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f9996-7323-42c2-95a3-8a2bbf72a4df None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:00.642099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:00.659490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a3337-a57f-4c4b-b29e-0f491f4002c9 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:00.666935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:00.696017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3d3f6-3059-4888-8229-ad7327bf8fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:00.696714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3d3f6-3059-4888-8229-ad7327bf8fb2 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:00.696999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3d3f6-3059-4888-8229-ad7327bf8fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:00.697277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3d3f6-3059-4888-8229-ad7327bf8fb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:00.698011 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3d3f6-3059-4888-8229-ad7327bf8fb2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:00.699658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3d3f6-3059-4888-8229-ad7327bf8fb2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:00.730487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 507/1016] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 26 20:52:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:52:00.748586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f9996-7323-42c2-95a3-8a2bbf72a4df None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:00.750465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57d2ac-b160-4c89-8f45-3964b49e6061 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:00.750852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57d2ac-b160-4c89-8f45-3964b49e6061 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:00.751132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57d2ac-b160-4c89-8f45-3964b49e6061 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:00.752741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 508/1017] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:00.759111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81cf4d-3525-4ad8-9fd1-4e06c99746be None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:00.759454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81cf4d-3525-4ad8-9fd1-4e06c99746be None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:00.759925 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81cf4d-3525-4ad8-9fd1-4e06c99746be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:00.777947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5e81cf4d-3525-4ad8-9fd1-4e06c99746be None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:00.779204 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5e81cf4d-3525-4ad8-9fd1-4e06c99746be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:00.798191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5919e-a5da-49aa-a9e8-b797554d11be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:00.798960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5919e-a5da-49aa-a9e8-b797554d11be None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:00.799247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5919e-a5da-49aa-a9e8-b797554d11be None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:00.799526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5919e-a5da-49aa-a9e8-b797554d11be None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:00.803445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5919e-a5da-49aa-a9e8-b797554d11be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:00.803445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5919e-a5da-49aa-a9e8-b797554d11be None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:00.832328 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 510/1018] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:52:00 2022] POST /identity/v3/projects => generated 311 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:52:00.869703 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 509/1019] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:00 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:52:00.893942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5919e-a5da-49aa-a9e8-b797554d11be None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:00.959459 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:01.010738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5919e-a5da-49aa-a9e8-b797554d11be None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:01.047951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee035a1-38fa-488c-998f-c490d2d90bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:01.048626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee035a1-38fa-488c-998f-c490d2d90bf6 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:01.048920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee035a1-38fa-488c-998f-c490d2d90bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:01.049202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee035a1-38fa-488c-998f-c490d2d90bf6 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:01.049945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee035a1-38fa-488c-998f-c490d2d90bf6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:01.051353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee035a1-38fa-488c-998f-c490d2d90bf6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:01.066898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 511/1020] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:52:00 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:52:03.056571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf92095-a005-478f-9194-85fcecb2c82c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.057134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf92095-a005-478f-9194-85fcecb2c82c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.057134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf92095-a005-478f-9194-85fcecb2c82c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.058748 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 510/1021] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:03.064978 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b6fad-aa60-4836-83f8-34757049dcb9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.065943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b6fad-aa60-4836-83f8-34757049dcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.066500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b6fad-aa60-4836-83f8-34757049dcb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.088575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ef0b6fad-aa60-4836-83f8-34757049dcb9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:03.090366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ef0b6fad-aa60-4836-83f8-34757049dcb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:03.099003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ff404-6fac-4b17-91d1-af04626ee6f1 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.099689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ff404-6fac-4b17-91d1-af04626ee6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.100679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ff404-6fac-4b17-91d1-af04626ee6f1 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.102510 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 511/1022] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:03.111686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67b68c-1a23-4b12-896c-9b2a28af3eec None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.112398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67b68c-1a23-4b12-896c-9b2a28af3eec None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.113588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67b68c-1a23-4b12-896c-9b2a28af3eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.136823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7b67b68c-1a23-4b12-896c-9b2a28af3eec None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:03.138294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7b67b68c-1a23-4b12-896c-9b2a28af3eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:03.175126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 512/1023] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:52:03.198300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b6fad-aa60-4836-83f8-34757049dcb9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:03.218143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 512/1024] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:03.243374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67b68c-1a23-4b12-896c-9b2a28af3eec None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:03.245410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:03.267565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:03.304555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b6fad-aa60-4836-83f8-34757049dcb9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:03.338050 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3ab4d-ebff-473f-8043-b00b437bd667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:03.338827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ab4d-ebff-473f-8043-b00b437bd667 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.339116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ab4d-ebff-473f-8043-b00b437bd667 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.339397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3ab4d-ebff-473f-8043-b00b437bd667 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.341001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67b68c-1a23-4b12-896c-9b2a28af3eec None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:03.349438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3ab4d-ebff-473f-8043-b00b437bd667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-23c3ab4d-ebff-473f-8043-b00b437bd667 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:03.358139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:03.359142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:03.360002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 513/1025] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 26 20:52:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:03.360715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10577833-e47e-4962-a63b-e92a321e24fd None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.360974 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10577833-e47e-4962-a63b-e92a321e24fd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.361241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10577833-e47e-4962-a63b-e92a321e24fd None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.362495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 514/1026] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:03.365080 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10577833-e47e-4962-a63b-e92a321e24fd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:03.378714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e16e2c-8c13-453f-a84f-c6fc7eee432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:03.379498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e16e2c-8c13-453f-a84f-c6fc7eee432f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.379779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e16e2c-8c13-453f-a84f-c6fc7eee432f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.380056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e16e2c-8c13-453f-a84f-c6fc7eee432f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.381239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e16e2c-8c13-453f-a84f-c6fc7eee432f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:03.382698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e16e2c-8c13-453f-a84f-c6fc7eee432f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:03.384286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 513/1027] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:03.385653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb8e57-e6f8-41d8-b02a-90e39baa0669 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.385943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb8e57-e6f8-41d8-b02a-90e39baa0669 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.386217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb8e57-e6f8-41d8-b02a-90e39baa0669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.402124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-31bb8e57-e6f8-41d8-b02a-90e39baa0669 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:03.403326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-31bb8e57-e6f8-41d8-b02a-90e39baa0669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:03.465975 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10577833-e47e-4962-a63b-e92a321e24fd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:03.467607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:03.469905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 514/1028] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:03.470758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb8e57-e6f8-41d8-b02a-90e39baa0669 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:03.494283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:03.508112 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f477d-a4e1-4baa-9e37-6edb7b237f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:03.508783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f477d-a4e1-4baa-9e37-6edb7b237f69 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.509069 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f477d-a4e1-4baa-9e37-6edb7b237f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.509351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f477d-a4e1-4baa-9e37-6edb7b237f69 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.510358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f477d-a4e1-4baa-9e37-6edb7b237f69 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:03.510699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f477d-a4e1-4baa-9e37-6edb7b237f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:03.512067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58f477d-a4e1-4baa-9e37-6edb7b237f69 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:03.520947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 515/1029] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 26 20:52:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:03.522364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f477d-a4e1-4baa-9e37-6edb7b237f69 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:03.581838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb8e57-e6f8-41d8-b02a-90e39baa0669 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:03.616166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d2c49-00cf-450e-9a2e-b08c286e76de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:03.616755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d2c49-00cf-450e-9a2e-b08c286e76de None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.617040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d2c49-00cf-450e-9a2e-b08c286e76de None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.617321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d2c49-00cf-450e-9a2e-b08c286e76de None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.618017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d2c49-00cf-450e-9a2e-b08c286e76de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:03.619495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d2c49-00cf-450e-9a2e-b08c286e76de None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:03.620778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58f477d-a4e1-4baa-9e37-6edb7b237f69 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:03.640813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 515/1030] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:52:03 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:52:03.641734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d2c49-00cf-450e-9a2e-b08c286e76de None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:03.656460 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcb2f1-b16b-47bc-a55f-3f7e2cfc99d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:03.657283 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcb2f1-b16b-47bc-a55f-3f7e2cfc99d3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.657570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcb2f1-b16b-47bc-a55f-3f7e2cfc99d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.657861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcb2f1-b16b-47bc-a55f-3f7e2cfc99d3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.658666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcb2f1-b16b-47bc-a55f-3f7e2cfc99d3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:03.659990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcb2f1-b16b-47bc-a55f-3f7e2cfc99d3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-40fcb2f1-b16b-47bc-a55f-3f7e2cfc99d3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:03.672836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:03.673967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:03.675494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 516/1031] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 26 20:52:03 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:03.680549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcb2f1-b16b-47bc-a55f-3f7e2cfc99d3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:03.714964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:03.747556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:03.777952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d2c49-00cf-450e-9a2e-b08c286e76de None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:03.809700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcb2f1-b16b-47bc-a55f-3f7e2cfc99d3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:03.819487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce995881-b22b-483e-b04f-ee2517453d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:03.820312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce995881-b22b-483e-b04f-ee2517453d67 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.820578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce995881-b22b-483e-b04f-ee2517453d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.820858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce995881-b22b-483e-b04f-ee2517453d67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.833298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce995881-b22b-483e-b04f-ee2517453d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:03.834849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce995881-b22b-483e-b04f-ee2517453d67 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:03.850973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527ce17-7bfa-489d-8466-003ff21e4c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:03.850973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527ce17-7bfa-489d-8466-003ff21e4c78 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:03.852159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527ce17-7bfa-489d-8466-003ff21e4c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:03.852159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527ce17-7bfa-489d-8466-003ff21e4c78 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:03.852159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527ce17-7bfa-489d-8466-003ff21e4c78 None admin] RBAC: Adding query filter params (name=image) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:03.852358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527ce17-7bfa-489d-8466-003ff21e4c78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:03.853725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527ce17-7bfa-489d-8466-003ff21e4c78 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-ce995881-b22b-483e-b04f-ee2517453d67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:52:03.858143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:03.859154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:03.860270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:52:03.860270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:52:03.860270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:03.860270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:03.860726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 517/1032] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:03.860850 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 516/1033] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:03.865757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527ce17-7bfa-489d-8466-003ff21e4c78 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:03.866380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce995881-b22b-483e-b04f-ee2517453d67 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:03.972724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:03.983686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527ce17-7bfa-489d-8466-003ff21e4c78 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:03.988936 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce995881-b22b-483e-b04f-ee2517453d67 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:04.015745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:04.024675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aec86-b2fb-4eb0-8b09-693ee1b16203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:04.025321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aec86-b2fb-4eb0-8b09-693ee1b16203 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:04.025605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aec86-b2fb-4eb0-8b09-693ee1b16203 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:04.025887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aec86-b2fb-4eb0-8b09-693ee1b16203 None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:04.026633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aec86-b2fb-4eb0-8b09-693ee1b16203 None admin] RBAC: Adding query filter params (type=image) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:04.026944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aec86-b2fb-4eb0-8b09-693ee1b16203 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:04.028326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aec86-b2fb-4eb0-8b09-693ee1b16203 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:04.028912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88393953-b6e0-4285-bd32-bbceb7f7cb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:04.029738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88393953-b6e0-4285-bd32-bbceb7f7cb4e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:04.030173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88393953-b6e0-4285-bd32-bbceb7f7cb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:04.030797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88393953-b6e0-4285-bd32-bbceb7f7cb4e None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:04.031666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88393953-b6e0-4285-bd32-bbceb7f7cb4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:04.032132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88393953-b6e0-4285-bd32-bbceb7f7cb4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:04.033324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 517/1034] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:04.033842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88393953-b6e0-4285-bd32-bbceb7f7cb4e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:04.040457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aec86-b2fb-4eb0-8b09-693ee1b16203 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:04.050569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 518/1035] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:04.058625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88393953-b6e0-4285-bd32-bbceb7f7cb4e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:04.144002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aec86-b2fb-4eb0-8b09-693ee1b16203 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:04.164041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88393953-b6e0-4285-bd32-bbceb7f7cb4e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:04.181717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136fd8d-ba05-4406-9e9a-c6cb24e1f347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:04.182380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136fd8d-ba05-4406-9e9a-c6cb24e1f347 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:04.182704 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136fd8d-ba05-4406-9e9a-c6cb24e1f347 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:04.182987 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136fd8d-ba05-4406-9e9a-c6cb24e1f347 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:04.183811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136fd8d-ba05-4406-9e9a-c6cb24e1f347 None admin] RBAC: Adding query filter params (interface=public, service_id=699ce898e84343f5a9035c1462af2eeb, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:04.184128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136fd8d-ba05-4406-9e9a-c6cb24e1f347 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:04.185514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136fd8d-ba05-4406-9e9a-c6cb24e1f347 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:04.190954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 518/1036] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 26 20:52:04 2022] GET /identity/v3/endpoints?service_id=699ce898e84343f5a9035c1462af2eeb&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:04.197028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136fd8d-ba05-4406-9e9a-c6cb24e1f347 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:04.205933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0170f-68e9-4b14-b5ab-0beb9842f844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:04.207353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0170f-68e9-4b14-b5ab-0beb9842f844 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:04.207643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0170f-68e9-4b14-b5ab-0beb9842f844 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:04.207925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0170f-68e9-4b14-b5ab-0beb9842f844 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:04.217002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0170f-68e9-4b14-b5ab-0beb9842f844 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:04.218489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0170f-68e9-4b14-b5ab-0beb9842f844 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-5ad0170f-68e9-4b14-b5ab-0beb9842f844 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:52:04.233833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:04.239703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 519/1037] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 26 20:52:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:04.240928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0170f-68e9-4b14-b5ab-0beb9842f844 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:04.255116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:04.265164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:04.311892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136fd8d-ba05-4406-9e9a-c6cb24e1f347 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:04.348384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c5d48-9980-4b4b-9f95-0e5b03df0cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:04.348558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0170f-68e9-4b14-b5ab-0beb9842f844 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:04.349055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c5d48-9980-4b4b-9f95-0e5b03df0cce None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:04.349343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c5d48-9980-4b4b-9f95-0e5b03df0cce None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:04.349658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c5d48-9980-4b4b-9f95-0e5b03df0cce None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:04.350335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c5d48-9980-4b4b-9f95-0e5b03df0cce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:04.351813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c5d48-9980-4b4b-9f95-0e5b03df0cce None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:04.358657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 519/1038] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 26 20:52:04 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:52:04.386332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e6da5-3352-4487-a83f-8714368dc9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:04.387076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e6da5-3352-4487-a83f-8714368dc9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:04.387363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e6da5-3352-4487-a83f-8714368dc9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:04.387641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e6da5-3352-4487-a83f-8714368dc9f5 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:04.388504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e6da5-3352-4487-a83f-8714368dc9f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:04.388820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e6da5-3352-4487-a83f-8714368dc9f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:04.390313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e6da5-3352-4487-a83f-8714368dc9f5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:04.415578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 520/1039] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 26 20:52:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:04.416746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c5d48-9980-4b4b-9f95-0e5b03df0cce None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:04.525921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c5d48-9980-4b4b-9f95-0e5b03df0cce None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:04.547422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:04.565167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e4874-41ba-4b0e-8b0e-36505731446f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:04.565824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e4874-41ba-4b0e-8b0e-36505731446f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:04.566108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e4874-41ba-4b0e-8b0e-36505731446f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:04.566386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e4874-41ba-4b0e-8b0e-36505731446f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:04.636038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e4874-41ba-4b0e-8b0e-36505731446f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:04.637491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e4874-41ba-4b0e-8b0e-36505731446f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:04.738905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 520/1040] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:04 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:52:04.740815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e6da5-3352-4487-a83f-8714368dc9f5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:04.819294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:04.852433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e6da5-3352-4487-a83f-8714368dc9f5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:04.887944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e86ee-6ffa-4f7a-9584-1ead327c15b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:04.888749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e86ee-6ffa-4f7a-9584-1ead327c15b8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:04.889032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e86ee-6ffa-4f7a-9584-1ead327c15b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:04.889304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e86ee-6ffa-4f7a-9584-1ead327c15b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:04.899666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e86ee-6ffa-4f7a-9584-1ead327c15b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:04.901113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e86ee-6ffa-4f7a-9584-1ead327c15b8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-031e86ee-6ffa-4f7a-9584-1ead327c15b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:52:04.918687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:04.919645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:04.920576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:52:04.920576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:52:04.920576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:04.920576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:04.920576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 521/1041] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:04.925396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e4874-41ba-4b0e-8b0e-36505731446f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:04.948331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:05.028057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e4874-41ba-4b0e-8b0e-36505731446f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:05.061898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fbb90-3bb7-4aa7-82e6-1b391f76ae77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:05.062580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fbb90-3bb7-4aa7-82e6-1b391f76ae77 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:05.062867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fbb90-3bb7-4aa7-82e6-1b391f76ae77 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:05.063143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fbb90-3bb7-4aa7-82e6-1b391f76ae77 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:05.063859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fbb90-3bb7-4aa7-82e6-1b391f76ae77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:05.064168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fbb90-3bb7-4aa7-82e6-1b391f76ae77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:05.065548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fbb90-3bb7-4aa7-82e6-1b391f76ae77 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:05.081148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 521/1042] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:05.082698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e86ee-6ffa-4f7a-9584-1ead327c15b8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:05.184035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e86ee-6ffa-4f7a-9584-1ead327c15b8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:05.206471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:05.219193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4f87e-61b4-4581-9562-d34da15a7c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:05.219990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4f87e-61b4-4581-9562-d34da15a7c64 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:05.220276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4f87e-61b4-4581-9562-d34da15a7c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:05.220556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4f87e-61b4-4581-9562-d34da15a7c64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:05.227803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4f87e-61b4-4581-9562-d34da15a7c64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:05.229260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4f87e-61b4-4581-9562-d34da15a7c64 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-42d4f87e-61b4-4581-9562-d34da15a7c64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:52:05.246156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:05.247256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:05.248180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:52:05.248180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:52:05.248180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:52:05.248180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:52:05.248180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:52:05.248180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:52:05.248180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:52:05.248180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:05.249131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 522/1043] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:52:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:05.253251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fbb90-3bb7-4aa7-82e6-1b391f76ae77 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:05.336425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:05.356883 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fbb90-3bb7-4aa7-82e6-1b391f76ae77 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:05.389711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d77b8-49a4-4f6b-a9c8-39dffb4a2347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:05.390373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d77b8-49a4-4f6b-a9c8-39dffb4a2347 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:05.390681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d77b8-49a4-4f6b-a9c8-39dffb4a2347 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:05.390958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d77b8-49a4-4f6b-a9c8-39dffb4a2347 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:05.391739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d77b8-49a4-4f6b-a9c8-39dffb4a2347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:05.392049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d77b8-49a4-4f6b-a9c8-39dffb4a2347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:05.393497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d77b8-49a4-4f6b-a9c8-39dffb4a2347 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:05.406095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 522/1044] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:52:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:05.411806 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4f87e-61b4-4581-9562-d34da15a7c64 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:05.515546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4f87e-61b4-4581-9562-d34da15a7c64 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:05.552870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc95c51-7505-4878-bc4d-97f95fe3dc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:05.553529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc95c51-7505-4878-bc4d-97f95fe3dc16 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:05.553815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc95c51-7505-4878-bc4d-97f95fe3dc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:05.554093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc95c51-7505-4878-bc4d-97f95fe3dc16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:05.554956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc95c51-7505-4878-bc4d-97f95fe3dc16 None admin] RBAC: Adding query filter params (role.id=190ab872a79449cdbc3d501f21d56d63, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=490d773bf21446789b77c5793a518a5a) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:05.555271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc95c51-7505-4878-bc4d-97f95fe3dc16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:05.556697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc95c51-7505-4878-bc4d-97f95fe3dc16 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:05.562496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 523/1045] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:52:05 2022] GET /identity/v3/role_assignments?user.id=490d773bf21446789b77c5793a518a5a&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=190ab872a79449cdbc3d501f21d56d63 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:05.948793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c1603-32be-4ef7-a2f7-75f13b39062a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:05.949115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c1603-32be-4ef7-a2f7-75f13b39062a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:05.949353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c1603-32be-4ef7-a2f7-75f13b39062a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:05.950876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 523/1046] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:05.955993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f318ee-a77a-49d6-80cb-8167e776d75e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:05.956358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f318ee-a77a-49d6-80cb-8167e776d75e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:05.956744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f318ee-a77a-49d6-80cb-8167e776d75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:05.974507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-61f318ee-a77a-49d6-80cb-8167e776d75e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:05.975681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-61f318ee-a77a-49d6-80cb-8167e776d75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:06.019486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:06.050489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 524/1047] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:52:06.072016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c1603-32be-4ef7-a2f7-75f13b39062a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:06.194837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c1603-32be-4ef7-a2f7-75f13b39062a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:06.229227 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158601a-a838-4a6a-8c7b-fde1cd13d12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:06.230023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158601a-a838-4a6a-8c7b-fde1cd13d12d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:06.230303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158601a-a838-4a6a-8c7b-fde1cd13d12d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:06.230606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158601a-a838-4a6a-8c7b-fde1cd13d12d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:06.238341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:06.243074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158601a-a838-4a6a-8c7b-fde1cd13d12d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f158601a-a838-4a6a-8c7b-fde1cd13d12d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:06.250845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:06.251789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:06.258687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 524/1048] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:52:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:06.261609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f318ee-a77a-49d6-80cb-8167e776d75e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:06.358865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f318ee-a77a-49d6-80cb-8167e776d75e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:06.396027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed37b19-97d2-4f67-b49d-80205499c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:06.396809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed37b19-97d2-4f67-b49d-80205499c6ee None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:06.397096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed37b19-97d2-4f67-b49d-80205499c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:06.397373 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed37b19-97d2-4f67-b49d-80205499c6ee None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:06.398131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed37b19-97d2-4f67-b49d-80205499c6ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:06.398439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed37b19-97d2-4f67-b49d-80205499c6ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:06.399868 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed37b19-97d2-4f67-b49d-80205499c6ee None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:06.413861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 525/1049] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:52:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:06.415696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158601a-a838-4a6a-8c7b-fde1cd13d12d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:06.516681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158601a-a838-4a6a-8c7b-fde1cd13d12d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:06.549437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75dd3e-4da2-495d-9922-5a9eaf44892d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:06.550309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75dd3e-4da2-495d-9922-5a9eaf44892d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:06.550623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75dd3e-4da2-495d-9922-5a9eaf44892d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:06.550901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75dd3e-4da2-495d-9922-5a9eaf44892d None admin] PATH_INFO: `/v3/users/beb974539d7045debf867849e35bfa70` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:06.565404 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75dd3e-4da2-495d-9922-5a9eaf44892d None admin] RBAC: Authorizing `identity:get_user(user_id=beb974539d7045debf867849e35bfa70)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:06.566910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75dd3e-4da2-495d-9922-5a9eaf44892d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:06.568983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 525/1050] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:06 2022] GET /identity/v3/users/beb974539d7045debf867849e35bfa70 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:06.573947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a238854-0ff3-4f18-b7d6-ae85a04ec68c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:06.574243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a238854-0ff3-4f18-b7d6-ae85a04ec68c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:06.574553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a238854-0ff3-4f18-b7d6-ae85a04ec68c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:06.575259 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75dd3e-4da2-495d-9922-5a9eaf44892d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:06.575906 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 526/1051] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:52:06.580696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ef0e3-cf34-488d-8377-c3b8adfc747e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:06.580984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ef0e3-cf34-488d-8377-c3b8adfc747e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:06.581262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ef0e3-cf34-488d-8377-c3b8adfc747e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:06.598182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e91ef0e3-cf34-488d-8377-c3b8adfc747e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:06.599037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e91ef0e3-cf34-488d-8377-c3b8adfc747e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:06.629490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:06.640319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:06.670794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 527/1052] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:52:06.692350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75dd3e-4da2-495d-9922-5a9eaf44892d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:06.693875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ef0e3-cf34-488d-8377-c3b8adfc747e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:06.728568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16d8ba-ca0a-443c-ad07-2e9459c0cc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:06.729226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16d8ba-ca0a-443c-ad07-2e9459c0cc18 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:06.729512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16d8ba-ca0a-443c-ad07-2e9459c0cc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:06.729788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16d8ba-ca0a-443c-ad07-2e9459c0cc18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:06.801853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16d8ba-ca0a-443c-ad07-2e9459c0cc18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:06.802043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ef0e3-cf34-488d-8377-c3b8adfc747e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:06.802126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16d8ba-ca0a-443c-ad07-2e9459c0cc18 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:06.840984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3c8fc-a18e-4131-8b82-36909589c820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:06.842095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3c8fc-a18e-4131-8b82-36909589c820 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:06.842618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3c8fc-a18e-4131-8b82-36909589c820 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:06.843096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3c8fc-a18e-4131-8b82-36909589c820 None admin] PATH_INFO: `/v3/services/image` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:06.843939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3c8fc-a18e-4131-8b82-36909589c820 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:06.845494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3c8fc-a18e-4131-8b82-36909589c820 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-70c3c8fc-a18e-4131-8b82-36909589c820 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:06.857884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:06.858951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 20:52:06.859965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:06.862508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 528/1053] 10.222.0.57 () {56 vars in 1105 bytes} [Thu May 26 20:52:06 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:06.864409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:06.868386 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3c8fc-a18e-4131-8b82-36909589c820 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:06.902849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:06.916255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 526/1054] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:06 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 20:52:06.917287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16d8ba-ca0a-443c-ad07-2e9459c0cc18 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:06.976750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3c8fc-a18e-4131-8b82-36909589c820 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:07.012727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd824bd-684b-4abc-b5e3-abb0c14a99b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:07.013389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd824bd-684b-4abc-b5e3-abb0c14a99b9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:07.013680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd824bd-684b-4abc-b5e3-abb0c14a99b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:07.014205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd824bd-684b-4abc-b5e3-abb0c14a99b9 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:07.014686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd824bd-684b-4abc-b5e3-abb0c14a99b9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:07.014999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd824bd-684b-4abc-b5e3-abb0c14a99b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:07.016358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd824bd-684b-4abc-b5e3-abb0c14a99b9 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:07.020270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16d8ba-ca0a-443c-ad07-2e9459c0cc18 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:07.022081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 529/1055] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:07.027429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd824bd-684b-4abc-b5e3-abb0c14a99b9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:07.059591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99696485-8ab7-46e0-b4b4-9cbb74aabf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:07.060446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99696485-8ab7-46e0-b4b4-9cbb74aabf8a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:07.060725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99696485-8ab7-46e0-b4b4-9cbb74aabf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:07.061007 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99696485-8ab7-46e0-b4b4-9cbb74aabf8a None admin] PATH_INFO: `/v3/projects/9c1c61d756484d07b712e8557a541cd1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:07.062198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99696485-8ab7-46e0-b4b4-9cbb74aabf8a None admin] RBAC: Authorizing `identity:get_project(project_id=9c1c61d756484d07b712e8557a541cd1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:07.063675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99696485-8ab7-46e0-b4b4-9cbb74aabf8a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:07.066472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 527/1056] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:06 2022] GET /identity/v3/projects/9c1c61d756484d07b712e8557a541cd1 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:07.071186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99696485-8ab7-46e0-b4b4-9cbb74aabf8a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:07.100540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:07.137394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:07.144905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd824bd-684b-4abc-b5e3-abb0c14a99b9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:07.181074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffb5f5-2534-47e5-b1fe-42316b6e8e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:07.181616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99696485-8ab7-46e0-b4b4-9cbb74aabf8a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:07.181729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffb5f5-2534-47e5-b1fe-42316b6e8e77 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:07.181934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffb5f5-2534-47e5-b1fe-42316b6e8e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:07.182199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ffb5f5-2534-47e5-b1fe-42316b6e8e77 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:07.182934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ffb5f5-2534-47e5-b1fe-42316b6e8e77 None admin] RBAC: Adding query filter params (type=image) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:07.183405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ffb5f5-2534-47e5-b1fe-42316b6e8e77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:07.185533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ffb5f5-2534-47e5-b1fe-42316b6e8e77 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:07.196357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 530/1057] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:07.197313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffb5f5-2534-47e5-b1fe-42316b6e8e77 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:07.220019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de513e0-22e6-458b-9cbb-55dc9e4278c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:07.221114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de513e0-22e6-458b-9cbb-55dc9e4278c2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:07.221417 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de513e0-22e6-458b-9cbb-55dc9e4278c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:07.221951 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de513e0-22e6-458b-9cbb-55dc9e4278c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:07.223324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de513e0-22e6-458b-9cbb-55dc9e4278c2 None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=9c1c61d756484d07b712e8557a541cd1, user.id=beb974539d7045debf867849e35bfa70) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:07.224342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de513e0-22e6-458b-9cbb-55dc9e4278c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:07.226297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de513e0-22e6-458b-9cbb-55dc9e4278c2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:07.242487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 528/1058] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:52:07 2022] GET /identity/v3/role_assignments?user.id=beb974539d7045debf867849e35bfa70&scope.project.id=9c1c61d756484d07b712e8557a541cd1&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:07.306596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ffb5f5-2534-47e5-b1fe-42316b6e8e77 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:07.342857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29136d2c-e1b7-4ed8-989d-9377b49e586a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:07.343531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29136d2c-e1b7-4ed8-989d-9377b49e586a None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:07.343817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29136d2c-e1b7-4ed8-989d-9377b49e586a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:07.344098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29136d2c-e1b7-4ed8-989d-9377b49e586a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:07.344847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29136d2c-e1b7-4ed8-989d-9377b49e586a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:07.346225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29136d2c-e1b7-4ed8-989d-9377b49e586a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:07.367715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 531/1059] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 26 20:52:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:52:07.789300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93788-c98b-46d9-a200-2d8d58c437c2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:07.789839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93788-c98b-46d9-a200-2d8d58c437c2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:07.789839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c93788-c98b-46d9-a200-2d8d58c437c2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:07.791391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 529/1060] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:07.796479 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d8c0-bb3d-41ac-b20e-bd62a9a17e98 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:07.796847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d8c0-bb3d-41ac-b20e-bd62a9a17e98 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:07.797206 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330d8c0-bb3d-41ac-b20e-bd62a9a17e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:07.814554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9330d8c0-bb3d-41ac-b20e-bd62a9a17e98 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:07.815808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9330d8c0-bb3d-41ac-b20e-bd62a9a17e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:07.827398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:07.901337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 532/1061] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:07.925266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c93788-c98b-46d9-a200-2d8d58c437c2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:08.017612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:08.029802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c93788-c98b-46d9-a200-2d8d58c437c2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:08.063560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a4910-2f64-41e2-8c7d-5a505989dd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:08.064422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a4910-2f64-41e2-8c7d-5a505989dd9c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:08.064711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a4910-2f64-41e2-8c7d-5a505989dd9c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:08.064989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a4910-2f64-41e2-8c7d-5a505989dd9c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:08.076097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a4910-2f64-41e2-8c7d-5a505989dd9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-b47a4910-2f64-41e2-8c7d-5a505989dd9c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:08.085026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:08.085989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:08.091309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 530/1062] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 26 20:52:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:08.094004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330d8c0-bb3d-41ac-b20e-bd62a9a17e98 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:08.208074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330d8c0-bb3d-41ac-b20e-bd62a9a17e98 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:08.242269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83bf24-7b9a-44cd-892c-99a7e89de56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:08.242966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83bf24-7b9a-44cd-892c-99a7e89de56d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:08.243254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83bf24-7b9a-44cd-892c-99a7e89de56d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:08.243531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83bf24-7b9a-44cd-892c-99a7e89de56d None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:08.244271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83bf24-7b9a-44cd-892c-99a7e89de56d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:08.244580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83bf24-7b9a-44cd-892c-99a7e89de56d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:08.245943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83bf24-7b9a-44cd-892c-99a7e89de56d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:08.258316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 533/1063] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 26 20:52:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:08.259944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a4910-2f64-41e2-8c7d-5a505989dd9c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:08.359838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a4910-2f64-41e2-8c7d-5a505989dd9c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:08.393943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82f412-2c51-43b9-a0bd-e30eabafdd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:08.394755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82f412-2c51-43b9-a0bd-e30eabafdd74 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:08.395044 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82f412-2c51-43b9-a0bd-e30eabafdd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:08.395322 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82f412-2c51-43b9-a0bd-e30eabafdd74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:08.406405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82f412-2c51-43b9-a0bd-e30eabafdd74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:08.407891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82f412-2c51-43b9-a0bd-e30eabafdd74 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-cb82f412-2c51-43b9-a0bd-e30eabafdd74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:52:08.425394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:08.426293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:08.432049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:52:08.432049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:52:08.432049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:08.432049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:08.432049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 531/1064] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:08.432626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83bf24-7b9a-44cd-892c-99a7e89de56d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:08.436540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:08.536400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83bf24-7b9a-44cd-892c-99a7e89de56d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:08.570025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5dd2c-fe2e-4eec-8b99-6f8f8b47f00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:08.570715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5dd2c-fe2e-4eec-8b99-6f8f8b47f00b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:08.571000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5dd2c-fe2e-4eec-8b99-6f8f8b47f00b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:08.571276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5dd2c-fe2e-4eec-8b99-6f8f8b47f00b None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:08.571997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5dd2c-fe2e-4eec-8b99-6f8f8b47f00b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:08.572304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5dd2c-fe2e-4eec-8b99-6f8f8b47f00b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:08.573691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5dd2c-fe2e-4eec-8b99-6f8f8b47f00b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:08.586076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 534/1065] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:08.591459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82f412-2c51-43b9-a0bd-e30eabafdd74 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:08.594664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:08.694667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82f412-2c51-43b9-a0bd-e30eabafdd74 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:08.727215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c7ddf-41e6-4000-9fdb-dff2f9dc8dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:08.728001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c7ddf-41e6-4000-9fdb-dff2f9dc8dad None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:08.728286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c7ddf-41e6-4000-9fdb-dff2f9dc8dad None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:08.728566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c7ddf-41e6-4000-9fdb-dff2f9dc8dad None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:08.736744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c7ddf-41e6-4000-9fdb-dff2f9dc8dad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:08.738202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8c7ddf-41e6-4000-9fdb-dff2f9dc8dad None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-3f8c7ddf-41e6-4000-9fdb-dff2f9dc8dad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:52:08.752624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:08.753601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:08.754595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 532/1066] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 26 20:52:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:08.761585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5dd2c-fe2e-4eec-8b99-6f8f8b47f00b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:08.829664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:08.868209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5dd2c-fe2e-4eec-8b99-6f8f8b47f00b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:08.904006 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad4507-668b-4f66-bfd9-0168fcd6dffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:08.904674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad4507-668b-4f66-bfd9-0168fcd6dffd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:08.904954 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad4507-668b-4f66-bfd9-0168fcd6dffd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:08.905228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad4507-668b-4f66-bfd9-0168fcd6dffd None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:08.906043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad4507-668b-4f66-bfd9-0168fcd6dffd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:08.906351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad4507-668b-4f66-bfd9-0168fcd6dffd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:08.907795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad4507-668b-4f66-bfd9-0168fcd6dffd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:08.929678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 535/1067] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 26 20:52:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:08.930833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c7ddf-41e6-4000-9fdb-dff2f9dc8dad None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:08.992312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:09.032704 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c7ddf-41e6-4000-9fdb-dff2f9dc8dad None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:09.065765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f757-e0e6-46dd-823f-b6346ca7548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:09.066579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f757-e0e6-46dd-823f-b6346ca7548c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.066862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f757-e0e6-46dd-823f-b6346ca7548c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.067132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7f757-e0e6-46dd-823f-b6346ca7548c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.078475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7f757-e0e6-46dd-823f-b6346ca7548c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:09.079937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7f757-e0e6-46dd-823f-b6346ca7548c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8ad7f757-e0e6-46dd-823f-b6346ca7548c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:52:09.096684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:09.097715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:09.103237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:52:09.103237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:52:09.103237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:09.103237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:09.103237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 533/1068] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:09.103873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad4507-668b-4f66-bfd9-0168fcd6dffd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:09.202586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad4507-668b-4f66-bfd9-0168fcd6dffd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:09.225564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:09.243945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911b7bd-8f80-4c3a-a079-927b50b5961f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:09.244843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911b7bd-8f80-4c3a-a079-927b50b5961f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.245277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911b7bd-8f80-4c3a-a079-927b50b5961f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.245703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911b7bd-8f80-4c3a-a079-927b50b5961f None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.246624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911b7bd-8f80-4c3a-a079-927b50b5961f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:09.247077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911b7bd-8f80-4c3a-a079-927b50b5961f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:09.248581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911b7bd-8f80-4c3a-a079-927b50b5961f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:09.266383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 536/1069] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:09.267834 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f757-e0e6-46dd-823f-b6346ca7548c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:09.370426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7f757-e0e6-46dd-823f-b6346ca7548c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:09.384967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:09.405114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59505d-0293-4b59-b817-8ad0f143b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:09.405899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59505d-0293-4b59-b817-8ad0f143b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.406187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59505d-0293-4b59-b817-8ad0f143b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.406483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59505d-0293-4b59-b817-8ad0f143b3c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.413172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59505d-0293-4b59-b817-8ad0f143b3c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:09.414657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59505d-0293-4b59-b817-8ad0f143b3c1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-db59505d-0293-4b59-b817-8ad0f143b3c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:52:09.430243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:09.431252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:09.437441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:52:09.437441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:52:09.437441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:52:09.437441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:52:09.437441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:52:09.437441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:52:09.437441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:52:09.437441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:09.437441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 534/1070] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:52:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:09.439403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911b7bd-8f80-4c3a-a079-927b50b5961f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:09.463638 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd52745-04ae-47bd-8712-3cd029e34ad8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.463931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd52745-04ae-47bd-8712-3cd029e34ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.464216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd52745-04ae-47bd-8712-3cd029e34ad8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.465705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 535/1071] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:09.470593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc8584-dc7a-449d-aa86-0c00cf8178b1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.470885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc8584-dc7a-449d-aa86-0c00cf8178b1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.471190 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc8584-dc7a-449d-aa86-0c00cf8178b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.488594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-56fc8584-dc7a-449d-aa86-0c00cf8178b1 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:09.489775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-56fc8584-dc7a-449d-aa86-0c00cf8178b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:09.560836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911b7bd-8f80-4c3a-a079-927b50b5961f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:09.568559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 536/1072] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:52:09.590230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc8584-dc7a-449d-aa86-0c00cf8178b1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:09.613470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5f510-7b9c-4bb6-bcb3-e73b5f31844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:09.614490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f510-7b9c-4bb6-bcb3-e73b5f31844a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.615027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f510-7b9c-4bb6-bcb3-e73b5f31844a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.616028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5f510-7b9c-4bb6-bcb3-e73b5f31844a None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.617008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5f510-7b9c-4bb6-bcb3-e73b5f31844a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:09.617541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5f510-7b9c-4bb6-bcb3-e73b5f31844a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:09.619108 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5f510-7b9c-4bb6-bcb3-e73b5f31844a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:09.631449 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 537/1073] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:52:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:09.632735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c2c4-7173-4f58-9ef1-228f3625ce78 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.633025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c2c4-7173-4f58-9ef1-228f3625ce78 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.633305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360c2c4-7173-4f58-9ef1-228f3625ce78 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.634966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 538/1074] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:09.637413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c2c4-7173-4f58-9ef1-228f3625ce78 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:09.681377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:09.694437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc8584-dc7a-449d-aa86-0c00cf8178b1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:09.696730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:09.728193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3976e3-0960-4e9b-92c0-bc1724ecc851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:09.728980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3976e3-0960-4e9b-92c0-bc1724ecc851 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.729266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3976e3-0960-4e9b-92c0-bc1724ecc851 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.729549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3976e3-0960-4e9b-92c0-bc1724ecc851 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.738344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360c2c4-7173-4f58-9ef1-228f3625ce78 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:09.739517 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3976e3-0960-4e9b-92c0-bc1724ecc851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-eb3976e3-0960-4e9b-92c0-bc1724ecc851 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:09.747788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:09.748783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:09.749668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 537/1075] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:52:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:09.750859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8f504-501a-42c3-abe7-baf184ade830 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.750859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8f504-501a-42c3-abe7-baf184ade830 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.751017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8f504-501a-42c3-abe7-baf184ade830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.767410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3ce8f504-501a-42c3-abe7-baf184ade830 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:09.768778 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3ce8f504-501a-42c3-abe7-baf184ade830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:09.773523 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5713752-fda4-4666-984b-27baf9153709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:09.774400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5713752-fda4-4666-984b-27baf9153709 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.774710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5713752-fda4-4666-984b-27baf9153709 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.774984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5713752-fda4-4666-984b-27baf9153709 None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/490d773bf21446789b77c5793a518a5a/roles/190ab872a79449cdbc3d501f21d56d63` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.777175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5713752-fda4-4666-984b-27baf9153709 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=490d773bf21446789b77c5793a518a5a, role_id=190ab872a79449cdbc3d501f21d56d63)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:09.778621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5713752-fda4-4666-984b-27baf9153709 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:09.791642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 539/1076] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:52:09 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/490d773bf21446789b77c5793a518a5a/roles/190ab872a79449cdbc3d501f21d56d63 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:52:09.792410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5713752-fda4-4666-984b-27baf9153709 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:09.859184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 538/1077] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:09.878121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8f504-501a-42c3-abe7-baf184ade830 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:09.916013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5713752-fda4-4666-984b-27baf9153709 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:09.918194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:09.957128 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a1ff6-1190-43d5-b55b-4519578f41bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:09.957811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a1ff6-1190-43d5-b55b-4519578f41bd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:09.958105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a1ff6-1190-43d5-b55b-4519578f41bd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:09.958384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a1ff6-1190-43d5-b55b-4519578f41bd None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:09.959708 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a1ff6-1190-43d5-b55b-4519578f41bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:09.960025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a1ff6-1190-43d5-b55b-4519578f41bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:09.961419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a1ff6-1190-43d5-b55b-4519578f41bd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:09.971940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 540/1078] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:52:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:09.978491 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a1ff6-1190-43d5-b55b-4519578f41bd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:09.993419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:10.002343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8f504-501a-42c3-abe7-baf184ade830 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:10.040745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27864732-5ecc-465a-ac4f-31995cfcb2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:10.041745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27864732-5ecc-465a-ac4f-31995cfcb2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:10.042234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27864732-5ecc-465a-ac4f-31995cfcb2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:10.042680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27864732-5ecc-465a-ac4f-31995cfcb2b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:10.086293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a1ff6-1190-43d5-b55b-4519578f41bd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:10.115119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27864732-5ecc-465a-ac4f-31995cfcb2b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:10.116776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27864732-5ecc-465a-ac4f-31995cfcb2b7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:10.121766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd4bb6-847b-49bd-ad73-0d75ddeaa662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:10.122679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4bb6-847b-49bd-ad73-0d75ddeaa662 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:10.122967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4bb6-847b-49bd-ad73-0d75ddeaa662 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:10.123251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4bb6-847b-49bd-ad73-0d75ddeaa662 None admin] PATH_INFO: `/v3/users/beb974539d7045debf867849e35bfa70` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:10.124462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd4bb6-847b-49bd-ad73-0d75ddeaa662 None admin] RBAC: Authorizing `identity:get_user(user_id=beb974539d7045debf867849e35bfa70)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:10.125896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd4bb6-847b-49bd-ad73-0d75ddeaa662 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:10.132266 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 541/1079] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:09 2022] GET /identity/v3/users/beb974539d7045debf867849e35bfa70 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:10.133089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd4bb6-847b-49bd-ad73-0d75ddeaa662 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:10.165228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:10.213697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:10.222623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 539/1080] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:52:10.228449 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27864732-5ecc-465a-ac4f-31995cfcb2b7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:10.236032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd4bb6-847b-49bd-ad73-0d75ddeaa662 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:10.273654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4eef6-c48c-4133-808c-70b5d3eb77bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:10.275000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4eef6-c48c-4133-808c-70b5d3eb77bf None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:10.275484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4eef6-c48c-4133-808c-70b5d3eb77bf None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:10.275953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4eef6-c48c-4133-808c-70b5d3eb77bf None admin] PATH_INFO: `/v3/projects/9c1c61d756484d07b712e8557a541cd1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:10.277380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4eef6-c48c-4133-808c-70b5d3eb77bf None admin] RBAC: Authorizing `identity:get_project(project_id=9c1c61d756484d07b712e8557a541cd1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:10.279158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4eef6-c48c-4133-808c-70b5d3eb77bf None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:10.281534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 542/1081] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:10 2022] GET /identity/v3/projects/9c1c61d756484d07b712e8557a541cd1 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:10.287451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4eef6-c48c-4133-808c-70b5d3eb77bf None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:10.335347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27864732-5ecc-465a-ac4f-31995cfcb2b7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:10.369795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f31c4a-7c3c-4302-b288-6ca029a836c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:10.370600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f31c4a-7c3c-4302-b288-6ca029a836c5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:10.370885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f31c4a-7c3c-4302-b288-6ca029a836c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:10.371156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f31c4a-7c3c-4302-b288-6ca029a836c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:10.381800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f31c4a-7c3c-4302-b288-6ca029a836c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:10.383293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f31c4a-7c3c-4302-b288-6ca029a836c5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:10.393803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:10.397813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4eef6-c48c-4133-808c-70b5d3eb77bf None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-41f31c4a-7c3c-4302-b288-6ca029a836c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:52:10.401281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:10.402194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:10.403164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:52:10.403164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:52:10.403164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:10.403164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:10.403164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 540/1082] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:10.408272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f31c4a-7c3c-4302-b288-6ca029a836c5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:10.434697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655f0d4-8366-41a9-a2f8-57429621dda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:10.435581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655f0d4-8366-41a9-a2f8-57429621dda6 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:10.435865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655f0d4-8366-41a9-a2f8-57429621dda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:10.436149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655f0d4-8366-41a9-a2f8-57429621dda6 None admin] PATH_INFO: `/v3/projects/9c1c61d756484d07b712e8557a541cd1/users/beb974539d7045debf867849e35bfa70/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:10.438113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655f0d4-8366-41a9-a2f8-57429621dda6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c1c61d756484d07b712e8557a541cd1, user_id=beb974539d7045debf867849e35bfa70, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:10.439589 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655f0d4-8366-41a9-a2f8-57429621dda6 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:10.453522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 543/1083] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:52:10 2022] PUT /identity/v3/projects/9c1c61d756484d07b712e8557a541cd1/users/beb974539d7045debf867849e35bfa70/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:52:10.456528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:10.517638 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f31c4a-7c3c-4302-b288-6ca029a836c5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:10.555073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89119f8a-cca2-45f7-bb66-32bdf006b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:10.555757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89119f8a-cca2-45f7-bb66-32bdf006b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:10.556047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89119f8a-cca2-45f7-bb66-32bdf006b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:10.556306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89119f8a-cca2-45f7-bb66-32bdf006b4c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:10.557032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89119f8a-cca2-45f7-bb66-32bdf006b4c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:10.557339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89119f8a-cca2-45f7-bb66-32bdf006b4c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:10.558767 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89119f8a-cca2-45f7-bb66-32bdf006b4c9 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:10.578491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 541/1084] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:12.000309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938967e1-e92f-4275-969f-a4561cc1e129 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:12.000830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938967e1-e92f-4275-969f-a4561cc1e129 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:12.000924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938967e1-e92f-4275-969f-a4561cc1e129 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:12.002270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 544/1085] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:12.007504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b79ae2-02a4-42d8-adc8-1f3e7e5c2b8c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:12.007914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b79ae2-02a4-42d8-adc8-1f3e7e5c2b8c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:12.008073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b79ae2-02a4-42d8-adc8-1f3e7e5c2b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:12.024570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-86b79ae2-02a4-42d8-adc8-1f3e7e5c2b8c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:12.025852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-86b79ae2-02a4-42d8-adc8-1f3e7e5c2b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:12.091460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 542/1086] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:12.114624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938967e1-e92f-4275-969f-a4561cc1e129 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:12.213863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938967e1-e92f-4275-969f-a4561cc1e129 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:12.246881 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354c94-63b6-474c-87b9-25acafe94095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:12.247695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354c94-63b6-474c-87b9-25acafe94095 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:12.247983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354c94-63b6-474c-87b9-25acafe94095 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:12.248265 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354c94-63b6-474c-87b9-25acafe94095 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:12.258794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9354c94-63b6-474c-87b9-25acafe94095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-c9354c94-63b6-474c-87b9-25acafe94095 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:12.267124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:12.268138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:12.269019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 545/1087] 10.222.0.57 () {56 vars in 1120 bytes} [Thu May 26 20:52:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:12.274739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b79ae2-02a4-42d8-adc8-1f3e7e5c2b8c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:12.307824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:12.370945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b79ae2-02a4-42d8-adc8-1f3e7e5c2b8c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:12.403617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310ade4-21a4-4d8e-903c-b9f8c4e934cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:12.404275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310ade4-21a4-4d8e-903c-b9f8c4e934cb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:12.404556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310ade4-21a4-4d8e-903c-b9f8c4e934cb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:12.404845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310ade4-21a4-4d8e-903c-b9f8c4e934cb None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:12.405584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310ade4-21a4-4d8e-903c-b9f8c4e934cb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:12.405892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310ade4-21a4-4d8e-903c-b9f8c4e934cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:12.407302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310ade4-21a4-4d8e-903c-b9f8c4e934cb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:12.415574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 543/1088] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 26 20:52:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:12.421205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354c94-63b6-474c-87b9-25acafe94095 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:12.444741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:12.523607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354c94-63b6-474c-87b9-25acafe94095 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:12.555829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3080f4-01fa-4941-8720-89cba62849f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:12.556614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3080f4-01fa-4941-8720-89cba62849f4 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:12.556894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3080f4-01fa-4941-8720-89cba62849f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:12.557200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3080f4-01fa-4941-8720-89cba62849f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:12.567026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3080f4-01fa-4941-8720-89cba62849f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:12.569136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3080f4-01fa-4941-8720-89cba62849f4 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-ea3080f4-01fa-4941-8720-89cba62849f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:52:12.586733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:12.587772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:12.588684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:52:12.588684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:52:12.588684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:12.588684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:12.588684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 546/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:12.593566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310ade4-21a4-4d8e-903c-b9f8c4e934cb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:12.621377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cffa1b-0b9d-4067-9ea8-6d25cfd2ef5d None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:12.621592 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cffa1b-0b9d-4067-9ea8-6d25cfd2ef5d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:12.621872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cffa1b-0b9d-4067-9ea8-6d25cfd2ef5d None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:12.623290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 547/1090] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:52:12.628135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fd1f2-0558-4bcc-8004-b82aaae658b1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:12.628428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fd1f2-0558-4bcc-8004-b82aaae658b1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:12.628710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fd1f2-0558-4bcc-8004-b82aaae658b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:12.645644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6b7fd1f2-0558-4bcc-8004-b82aaae658b1 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:12.647465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6b7fd1f2-0558-4bcc-8004-b82aaae658b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:12.687467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:12.691684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310ade4-21a4-4d8e-903c-b9f8c4e934cb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:12.715225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 548/1091] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:52:12.726223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935634a2-1c34-44a5-913d-000770aa9850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:12.726911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935634a2-1c34-44a5-913d-000770aa9850 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:12.727199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935634a2-1c34-44a5-913d-000770aa9850 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:12.727475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935634a2-1c34-44a5-913d-000770aa9850 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:12.728188 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935634a2-1c34-44a5-913d-000770aa9850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:12.728494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935634a2-1c34-44a5-913d-000770aa9850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:12.729867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935634a2-1c34-44a5-913d-000770aa9850 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:12.738650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fd1f2-0558-4bcc-8004-b82aaae658b1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:12.740666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:12.741644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 544/1092] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:52:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:12.746806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935634a2-1c34-44a5-913d-000770aa9850 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:12.837280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fd1f2-0558-4bcc-8004-b82aaae658b1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:12.844537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935634a2-1c34-44a5-913d-000770aa9850 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:12.870790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90786ca6-6ec4-431e-b45b-857af76ff335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:12.871627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90786ca6-6ec4-431e-b45b-857af76ff335 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:12.871863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90786ca6-6ec4-431e-b45b-857af76ff335 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:12.872146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90786ca6-6ec4-431e-b45b-857af76ff335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:12.877636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6724-6dfa-4453-97e1-1097180e4ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:12.878422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6724-6dfa-4453-97e1-1097180e4ddd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:12.878770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6724-6dfa-4453-97e1-1097180e4ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:12.879055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6724-6dfa-4453-97e1-1097180e4ddd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:12.882854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90786ca6-6ec4-431e-b45b-857af76ff335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:12.886832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c6724-6dfa-4453-97e1-1097180e4ddd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:12.888293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c6724-6dfa-4453-97e1-1097180e4ddd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-90786ca6-6ec4-431e-b45b-857af76ff335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:12.890782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:12.891909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 549/1093] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:52:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:12.900995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90786ca6-6ec4-431e-b45b-857af76ff335 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-8a5c6724-6dfa-4453-97e1-1097180e4ddd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:52:12.903921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:12.906197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:12.907505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 545/1094] 10.222.0.57 () {56 vars in 1093 bytes} [Thu May 26 20:52:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:12.910672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6724-6dfa-4453-97e1-1097180e4ddd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:12.929739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:12.968788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:13.008939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90786ca6-6ec4-431e-b45b-857af76ff335 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:13.011483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6724-6dfa-4453-97e1-1097180e4ddd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:13.041577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40ad07-a6b9-4b98-b341-9f4243aaa3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:13.042245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ad07-a6b9-4b98-b341-9f4243aaa3cc None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:13.042565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ad07-a6b9-4b98-b341-9f4243aaa3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:13.042849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40ad07-a6b9-4b98-b341-9f4243aaa3cc None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:13.043594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40ad07-a6b9-4b98-b341-9f4243aaa3cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:13.043904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40ad07-a6b9-4b98-b341-9f4243aaa3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:13.044012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a915d6-e360-425a-aa66-6edd6664f826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:13.044649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a915d6-e360-425a-aa66-6edd6664f826 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:13.044937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a915d6-e360-425a-aa66-6edd6664f826 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:13.045217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a915d6-e360-425a-aa66-6edd6664f826 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:13.045329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40ad07-a6b9-4b98-b341-9f4243aaa3cc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:13.046047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a915d6-e360-425a-aa66-6edd6664f826 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:13.046364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a915d6-e360-425a-aa66-6edd6664f826 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:13.047824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a915d6-e360-425a-aa66-6edd6664f826 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:13.053866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 550/1095] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:52:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:13.059536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40ad07-a6b9-4b98-b341-9f4243aaa3cc None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:13.066159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 546/1096] 10.222.0.57 () {56 vars in 1133 bytes} [Thu May 26 20:52:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:13.072205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a915d6-e360-425a-aa66-6edd6664f826 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:13.148074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:13.159286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40ad07-a6b9-4b98-b341-9f4243aaa3cc None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:13.170830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a915d6-e360-425a-aa66-6edd6664f826 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:13.181917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:13.192793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993911f-c45b-4150-b2b2-e2cca94201b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:13.194218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993911f-c45b-4150-b2b2-e2cca94201b7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:13.194527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993911f-c45b-4150-b2b2-e2cca94201b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:13.194807 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993911f-c45b-4150-b2b2-e2cca94201b7 None admin] PATH_INFO: `/v3/users/beb974539d7045debf867849e35bfa70` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:13.195998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993911f-c45b-4150-b2b2-e2cca94201b7 None admin] RBAC: Authorizing `identity:get_user(user_id=beb974539d7045debf867849e35bfa70)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:13.197427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1993911f-c45b-4150-b2b2-e2cca94201b7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:13.199104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 551/1097] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:13 2022] GET /identity/v3/users/beb974539d7045debf867849e35bfa70 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:13.204727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993911f-c45b-4150-b2b2-e2cca94201b7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:13.207439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0eeca-989d-4ba7-ae38-626abd1ecfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:13.208088 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0eeca-989d-4ba7-ae38-626abd1ecfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:13.208372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0eeca-989d-4ba7-ae38-626abd1ecfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:13.208649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0eeca-989d-4ba7-ae38-626abd1ecfd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:13.275654 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0eeca-989d-4ba7-ae38-626abd1ecfd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:13.277097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0eeca-989d-4ba7-ae38-626abd1ecfd1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:13.304191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1993911f-c45b-4150-b2b2-e2cca94201b7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:13.337676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01a075-551a-4639-8536-b0ec13eec481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:13.338325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01a075-551a-4639-8536-b0ec13eec481 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:13.338642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01a075-551a-4639-8536-b0ec13eec481 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:13.338925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01a075-551a-4639-8536-b0ec13eec481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:13.351862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:13.375943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 547/1098] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:52:13.382001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0eeca-989d-4ba7-ae38-626abd1ecfd1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:13.398191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:13.408025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01a075-551a-4639-8536-b0ec13eec481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:13.409498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01a075-551a-4639-8536-b0ec13eec481 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:13.479145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0eeca-989d-4ba7-ae38-626abd1ecfd1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:13.505218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 552/1099] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 20:52:13.511672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01a075-551a-4639-8536-b0ec13eec481 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:13.518267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe86a7-36ef-4a7e-83a8-b2afe7ecc6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:13.519085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe86a7-36ef-4a7e-83a8-b2afe7ecc6bd None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:13.519364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe86a7-36ef-4a7e-83a8-b2afe7ecc6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:13.519639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe86a7-36ef-4a7e-83a8-b2afe7ecc6bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:13.529620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe86a7-36ef-4a7e-83a8-b2afe7ecc6bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:13.531111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe86a7-36ef-4a7e-83a8-b2afe7ecc6bd None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-bebe86a7-36ef-4a7e-83a8-b2afe7ecc6bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:52:13.548547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:13.549642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:13.550617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:52:13.550617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:52:13.550617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:52:13.550617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:13.550617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 548/1100] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:13.555779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe86a7-36ef-4a7e-83a8-b2afe7ecc6bd None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:13.582472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:13.600523 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:13.611961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01a075-551a-4639-8536-b0ec13eec481 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:13.645430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0faf830-b27b-4852-aa0a-edf0a488f36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:13.646818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faf830-b27b-4852-aa0a-edf0a488f36f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:13.647105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faf830-b27b-4852-aa0a-edf0a488f36f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:13.647380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0faf830-b27b-4852-aa0a-edf0a488f36f None admin] PATH_INFO: `/v3/projects/9c1c61d756484d07b712e8557a541cd1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:13.648603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0faf830-b27b-4852-aa0a-edf0a488f36f None admin] RBAC: Authorizing `identity:get_project(project_id=9c1c61d756484d07b712e8557a541cd1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:13.649990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0faf830-b27b-4852-aa0a-edf0a488f36f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:13.651716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 553/1101] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:13 2022] GET /identity/v3/projects/9c1c61d756484d07b712e8557a541cd1 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:13.657659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0faf830-b27b-4852-aa0a-edf0a488f36f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:13.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe86a7-36ef-4a7e-83a8-b2afe7ecc6bd None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:13.692603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86505fe1-7090-4195-addc-688ef5ac2cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:13.693259 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86505fe1-7090-4195-addc-688ef5ac2cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:13.693542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86505fe1-7090-4195-addc-688ef5ac2cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:13.693820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86505fe1-7090-4195-addc-688ef5ac2cf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:13.694564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86505fe1-7090-4195-addc-688ef5ac2cf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:13.694873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86505fe1-7090-4195-addc-688ef5ac2cf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:13.696262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86505fe1-7090-4195-addc-688ef5ac2cf7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:13.707190 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 549/1102] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:13.713047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86505fe1-7090-4195-addc-688ef5ac2cf7 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:13.757969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0faf830-b27b-4852-aa0a-edf0a488f36f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:13.792251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29cbaf-286b-40e2-9c47-0886128de196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:13.792902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29cbaf-286b-40e2-9c47-0886128de196 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:13.793188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29cbaf-286b-40e2-9c47-0886128de196 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:13.793465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29cbaf-286b-40e2-9c47-0886128de196 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:13.794281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29cbaf-286b-40e2-9c47-0886128de196 None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=9c1c61d756484d07b712e8557a541cd1, user.id=beb974539d7045debf867849e35bfa70) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:13.794628 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29cbaf-286b-40e2-9c47-0886128de196 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:13.796040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29cbaf-286b-40e2-9c47-0886128de196 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:13.797981 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:13.806370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 554/1103] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:52:13 2022] GET /identity/v3/role_assignments?user.id=beb974539d7045debf867849e35bfa70&scope.project.id=9c1c61d756484d07b712e8557a541cd1&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:13.810647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86505fe1-7090-4195-addc-688ef5ac2cf7 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:13.847757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6285afa-6484-4bf5-9da1-9860c1b1ac12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:13.848574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6285afa-6484-4bf5-9da1-9860c1b1ac12 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:13.848861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6285afa-6484-4bf5-9da1-9860c1b1ac12 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:13.849158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6285afa-6484-4bf5-9da1-9860c1b1ac12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:13.856709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6285afa-6484-4bf5-9da1-9860c1b1ac12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:13.858206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6285afa-6484-4bf5-9da1-9860c1b1ac12 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-f6285afa-6484-4bf5-9da1-9860c1b1ac12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:52:13.874954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:13.876071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:13.876998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:52:13.876998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:52:13.876998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:52:13.876998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:52:13.876998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:52:13.876998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:52:13.876998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:52:13.876998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:13.877714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 550/1104] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:52:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:52:13.882882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29cbaf-286b-40e2-9c47-0886128de196 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:13.909454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:13.987922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29cbaf-286b-40e2-9c47-0886128de196 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:14.022193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd07ed-f9fc-4c96-be9c-25c86b0aa2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:14.022891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd07ed-f9fc-4c96-be9c-25c86b0aa2ac None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:14.023184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd07ed-f9fc-4c96-be9c-25c86b0aa2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:14.023462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd07ed-f9fc-4c96-be9c-25c86b0aa2ac None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:14.024238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd07ed-f9fc-4c96-be9c-25c86b0aa2ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:14.024553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd07ed-f9fc-4c96-be9c-25c86b0aa2ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:14.025983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd07ed-f9fc-4c96-be9c-25c86b0aa2ac None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:14.038058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 555/1105] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:52:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:14.044339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6285afa-6484-4bf5-9da1-9860c1b1ac12 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:14.141286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6285afa-6484-4bf5-9da1-9860c1b1ac12 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:14.174039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc78bab-90f7-4c32-8d4e-96d168dc4397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:14.174649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc78bab-90f7-4c32-8d4e-96d168dc4397 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:14.174932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc78bab-90f7-4c32-8d4e-96d168dc4397 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:14.175206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc78bab-90f7-4c32-8d4e-96d168dc4397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:14.176026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc78bab-90f7-4c32-8d4e-96d168dc4397 None admin] RBAC: Adding query filter params (role.id=190ab872a79449cdbc3d501f21d56d63, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=490d773bf21446789b77c5793a518a5a) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:14.176335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc78bab-90f7-4c32-8d4e-96d168dc4397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:14.178385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc78bab-90f7-4c32-8d4e-96d168dc4397 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:14.184297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 551/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:52:14 2022] GET /identity/v3/role_assignments?user.id=490d773bf21446789b77c5793a518a5a&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=190ab872a79449cdbc3d501f21d56d63 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:16.055769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fcf05-a0c0-46b0-b4bf-93c80cec8e46 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:16.056289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fcf05-a0c0-46b0-b4bf-93c80cec8e46 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:16.056399 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fcf05-a0c0-46b0-b4bf-93c80cec8e46 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:16.057686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 556/1107] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:16.062888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640511c4-e757-4c2e-bb8b-3aca39442fb1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:16.063274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640511c4-e757-4c2e-bb8b-3aca39442fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:16.063532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640511c4-e757-4c2e-bb8b-3aca39442fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:16.081256 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-640511c4-e757-4c2e-bb8b-3aca39442fb1 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:16.082439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-640511c4-e757-4c2e-bb8b-3aca39442fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:16.090350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:16.150700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 552/1108] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:16.175008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fcf05-a0c0-46b0-b4bf-93c80cec8e46 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:16.259215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:16.275836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fcf05-a0c0-46b0-b4bf-93c80cec8e46 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:16.308340 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66436ea4-344f-4922-a61f-f386fc07a491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:16.309111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436ea4-344f-4922-a61f-f386fc07a491 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:16.309392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436ea4-344f-4922-a61f-f386fc07a491 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:16.309667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66436ea4-344f-4922-a61f-f386fc07a491 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:16.310833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66436ea4-344f-4922-a61f-f386fc07a491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:16.312252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66436ea4-344f-4922-a61f-f386fc07a491 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:16.313829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 557/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:16.319548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640511c4-e757-4c2e-bb8b-3aca39442fb1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:16.415113 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640511c4-e757-4c2e-bb8b-3aca39442fb1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:16.447768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0938b-27e6-4141-9134-d2e52557c2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:16.448419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0938b-27e6-4141-9134-d2e52557c2d5 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:16.448706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0938b-27e6-4141-9134-d2e52557c2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:16.448987 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0938b-27e6-4141-9134-d2e52557c2d5 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:16.449688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0938b-27e6-4141-9134-d2e52557c2d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:16.451153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0938b-27e6-4141-9134-d2e52557c2d5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:16.464482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:16.471017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 553/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:52:16 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:52:18.669337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a290e6-7414-4391-b3db-476bcfffd274 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:18.669899 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a290e6-7414-4391-b3db-476bcfffd274 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:18.669899 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a290e6-7414-4391-b3db-476bcfffd274 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:18.671310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 558/1111] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:18.676801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dab13-96a9-462e-a8b8-ba0f9661f813 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:18.677221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dab13-96a9-462e-a8b8-ba0f9661f813 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:18.677367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91dab13-96a9-462e-a8b8-ba0f9661f813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:18.694727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f91dab13-96a9-462e-a8b8-ba0f9661f813 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:18.695889 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f91dab13-96a9-462e-a8b8-ba0f9661f813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:18.758515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 554/1112] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:18.782157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a290e6-7414-4391-b3db-476bcfffd274 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:18.879670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a290e6-7414-4391-b3db-476bcfffd274 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:18.912207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c2741-7a3d-4d9a-89b7-e4860a233e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:18.913061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c2741-7a3d-4d9a-89b7-e4860a233e76 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:18.913343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c2741-7a3d-4d9a-89b7-e4860a233e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:18.913615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c2741-7a3d-4d9a-89b7-e4860a233e76 None admin] PATH_INFO: `/v3/roles/fa67cb2f211d433097f606ff029f863e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:18.915103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c2741-7a3d-4d9a-89b7-e4860a233e76 None admin] RBAC: Authorizing `identity:get_role(role_id=fa67cb2f211d433097f606ff029f863e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:18.916462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c2741-7a3d-4d9a-89b7-e4860a233e76 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:18.917807 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 559/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:18 2022] GET /identity/v3/roles/fa67cb2f211d433097f606ff029f863e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:18.923644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91dab13-96a9-462e-a8b8-ba0f9661f813 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:19.023386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91dab13-96a9-462e-a8b8-ba0f9661f813 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:19.047851 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:19.056304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61ede1-ab8e-48a2-862b-247f8e2af79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:19.057158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ede1-ab8e-48a2-862b-247f8e2af79d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:19.057441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ede1-ab8e-48a2-862b-247f8e2af79d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:19.057717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ede1-ab8e-48a2-862b-247f8e2af79d None admin] PATH_INFO: `/v3/users/d397240bd1774309a092ecee3f000184` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:19.070873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61ede1-ab8e-48a2-862b-247f8e2af79d None admin] RBAC: Authorizing `identity:get_user(user_id=d397240bd1774309a092ecee3f000184)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:19.072338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61ede1-ab8e-48a2-862b-247f8e2af79d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:19.073994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 555/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:18 2022] GET /identity/v3/users/d397240bd1774309a092ecee3f000184 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:19.079999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c2741-7a3d-4d9a-89b7-e4860a233e76 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:19.101450 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:19.177666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c2741-7a3d-4d9a-89b7-e4860a233e76 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:19.209671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186c38-60c7-4d20-9784-8d0278655e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:19.210306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186c38-60c7-4d20-9784-8d0278655e1a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:19.210626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186c38-60c7-4d20-9784-8d0278655e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:19.210907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4186c38-60c7-4d20-9784-8d0278655e1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:19.275412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4186c38-60c7-4d20-9784-8d0278655e1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:19.276861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4186c38-60c7-4d20-9784-8d0278655e1a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:19.297491 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:19.374178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 560/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:52:19.380135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61ede1-ab8e-48a2-862b-247f8e2af79d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:19.476336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61ede1-ab8e-48a2-862b-247f8e2af79d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:19.508825 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d9f3b-860d-4d2c-a4d0-6e9c7a2a377b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:19.509929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d9f3b-860d-4d2c-a4d0-6e9c7a2a377b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:19.510211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d9f3b-860d-4d2c-a4d0-6e9c7a2a377b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:19.510511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d9f3b-860d-4d2c-a4d0-6e9c7a2a377b None admin] PATH_INFO: `/v3/projects/9c1c61d756484d07b712e8557a541cd1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:19.511683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d9f3b-860d-4d2c-a4d0-6e9c7a2a377b None admin] RBAC: Authorizing `identity:get_project(project_id=9c1c61d756484d07b712e8557a541cd1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:19.513119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d9f3b-860d-4d2c-a4d0-6e9c7a2a377b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:19.514812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 556/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:19 2022] GET /identity/v3/projects/9c1c61d756484d07b712e8557a541cd1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:19.520861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186c38-60c7-4d20-9784-8d0278655e1a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:19.617118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4186c38-60c7-4d20-9784-8d0278655e1a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:19.638714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:19.653458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8437f80-1053-4a49-bf36-737a73c4b57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:19.654105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8437f80-1053-4a49-bf36-737a73c4b57c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:19.654393 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8437f80-1053-4a49-bf36-737a73c4b57c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:19.654703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8437f80-1053-4a49-bf36-737a73c4b57c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:19.655526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8437f80-1053-4a49-bf36-737a73c4b57c None admin] RBAC: Adding query filter params (role.id=fa67cb2f211d433097f606ff029f863e, scope.project.id=9c1c61d756484d07b712e8557a541cd1, user.id=d397240bd1774309a092ecee3f000184) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:19.655841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8437f80-1053-4a49-bf36-737a73c4b57c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:19.657268 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8437f80-1053-4a49-bf36-737a73c4b57c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:19.663231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 561/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:52:19 2022] GET /identity/v3/role_assignments?user.id=d397240bd1774309a092ecee3f000184&scope.project.id=9c1c61d756484d07b712e8557a541cd1&role.id=fa67cb2f211d433097f606ff029f863e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:21.791900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8b9fe-04ea-4a9a-9920-6372d781e133 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:21.792443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8b9fe-04ea-4a9a-9920-6372d781e133 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:21.792443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8b9fe-04ea-4a9a-9920-6372d781e133 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:21.793802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 557/1118] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:21.798862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfee39-0306-4776-b442-d3f9ee7bb220 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:21.799239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfee39-0306-4776-b442-d3f9ee7bb220 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:21.799419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfee39-0306-4776-b442-d3f9ee7bb220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:21.816472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4bcfee39-0306-4776-b442-d3f9ee7bb220 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:21.817665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4bcfee39-0306-4776-b442-d3f9ee7bb220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:21.881220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 562/1119] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:21.903885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8b9fe-04ea-4a9a-9920-6372d781e133 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:21.957590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:22.001263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8b9fe-04ea-4a9a-9920-6372d781e133 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:22.033989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5849a27-c1a4-4a5e-903d-6d036528f320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:22.034858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5849a27-c1a4-4a5e-903d-6d036528f320 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:22.035142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5849a27-c1a4-4a5e-903d-6d036528f320 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:22.035417 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5849a27-c1a4-4a5e-903d-6d036528f320 None admin] PATH_INFO: `/v3/roles/fa67cb2f211d433097f606ff029f863e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:22.037514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5849a27-c1a4-4a5e-903d-6d036528f320 None admin] RBAC: Authorizing `identity:get_role(role_id=fa67cb2f211d433097f606ff029f863e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:22.038930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5849a27-c1a4-4a5e-903d-6d036528f320 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:22.040273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 558/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:21 2022] GET /identity/v3/roles/fa67cb2f211d433097f606ff029f863e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:22.045992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfee39-0306-4776-b442-d3f9ee7bb220 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:22.139983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfee39-0306-4776-b442-d3f9ee7bb220 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:22.153560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:22.173629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92866e-46be-4827-85a7-66c1ffb420a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:22.174513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92866e-46be-4827-85a7-66c1ffb420a7 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:22.174936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92866e-46be-4827-85a7-66c1ffb420a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:22.175215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92866e-46be-4827-85a7-66c1ffb420a7 None admin] PATH_INFO: `/v3/users/d397240bd1774309a092ecee3f000184` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:22.176380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92866e-46be-4827-85a7-66c1ffb420a7 None admin] RBAC: Authorizing `identity:get_user(user_id=d397240bd1774309a092ecee3f000184)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:22.177816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92866e-46be-4827-85a7-66c1ffb420a7 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:22.179430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 563/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:22 2022] GET /identity/v3/users/d397240bd1774309a092ecee3f000184 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:22.184888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5849a27-c1a4-4a5e-903d-6d036528f320 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:22.280846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5849a27-c1a4-4a5e-903d-6d036528f320 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:22.302353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:22.313738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab556ec3-c6ef-4ecf-be86-9fc1133c2a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:22.314622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab556ec3-c6ef-4ecf-be86-9fc1133c2a65 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:22.314908 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab556ec3-c6ef-4ecf-be86-9fc1133c2a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:22.315186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab556ec3-c6ef-4ecf-be86-9fc1133c2a65 None admin] PATH_INFO: `/v3/projects/9c1c61d756484d07b712e8557a541cd1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:22.316353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab556ec3-c6ef-4ecf-be86-9fc1133c2a65 None admin] RBAC: Authorizing `identity:get_project(project_id=9c1c61d756484d07b712e8557a541cd1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:22.318299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab556ec3-c6ef-4ecf-be86-9fc1133c2a65 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:22.319977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 559/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:22 2022] GET /identity/v3/projects/9c1c61d756484d07b712e8557a541cd1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:22.325494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92866e-46be-4827-85a7-66c1ffb420a7 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:22.420841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92866e-46be-4827-85a7-66c1ffb420a7 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:22.453383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b28c5-2c63-4449-b76f-867e5ec76d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:22.454243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b28c5-2c63-4449-b76f-867e5ec76d11 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:22.454554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b28c5-2c63-4449-b76f-867e5ec76d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:22.454835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b28c5-2c63-4449-b76f-867e5ec76d11 None admin] PATH_INFO: `/v3/projects/9c1c61d756484d07b712e8557a541cd1/users/d397240bd1774309a092ecee3f000184/roles/fa67cb2f211d433097f606ff029f863e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:22.456978 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b28c5-2c63-4449-b76f-867e5ec76d11 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c1c61d756484d07b712e8557a541cd1, user_id=d397240bd1774309a092ecee3f000184, role_id=fa67cb2f211d433097f606ff029f863e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:22.458965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b28c5-2c63-4449-b76f-867e5ec76d11 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:22.470876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 564/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:52:22 2022] PUT /identity/v3/projects/9c1c61d756484d07b712e8557a541cd1/users/d397240bd1774309a092ecee3f000184/roles/fa67cb2f211d433097f606ff029f863e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:52:24.633098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19116586-0148-476b-84a8-547d170e4930 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:24.633624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19116586-0148-476b-84a8-547d170e4930 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:24.633624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19116586-0148-476b-84a8-547d170e4930 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:24.635313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 560/1124] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:24.640619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7fe33-4045-490c-b55e-27f28fb8f8b9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:24.641033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7fe33-4045-490c-b55e-27f28fb8f8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:24.641188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb7fe33-4045-490c-b55e-27f28fb8f8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:24.657503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-abb7fe33-4045-490c-b55e-27f28fb8f8b9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:24.658709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-abb7fe33-4045-490c-b55e-27f28fb8f8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:24.696443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:24.737661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 565/1125] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:24.761923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19116586-0148-476b-84a8-547d170e4930 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:24.858956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19116586-0148-476b-84a8-547d170e4930 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:24.890999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856290c-3ef1-4e5b-a320-17ab1f90e1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:24.891783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856290c-3ef1-4e5b-a320-17ab1f90e1ac None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:24.892068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856290c-3ef1-4e5b-a320-17ab1f90e1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:24.892346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856290c-3ef1-4e5b-a320-17ab1f90e1ac None admin] PATH_INFO: `/v3/roles/fa67cb2f211d433097f606ff029f863e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:24.893806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856290c-3ef1-4e5b-a320-17ab1f90e1ac None admin] RBAC: Authorizing `identity:get_role(role_id=fa67cb2f211d433097f606ff029f863e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:24.895253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856290c-3ef1-4e5b-a320-17ab1f90e1ac None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:24.897116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 561/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:24 2022] GET /identity/v3/roles/fa67cb2f211d433097f606ff029f863e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:24.902804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7fe33-4045-490c-b55e-27f28fb8f8b9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:24.997752 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb7fe33-4045-490c-b55e-27f28fb8f8b9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:25.037104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4430d0d-5676-4203-a050-c176302002a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:25.037972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4430d0d-5676-4203-a050-c176302002a8 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:25.038261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4430d0d-5676-4203-a050-c176302002a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:25.038570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4430d0d-5676-4203-a050-c176302002a8 None admin] PATH_INFO: `/v3/users/d397240bd1774309a092ecee3f000184` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:25.039783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4430d0d-5676-4203-a050-c176302002a8 None admin] RBAC: Authorizing `identity:get_user(user_id=d397240bd1774309a092ecee3f000184)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:25.041234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4430d0d-5676-4203-a050-c176302002a8 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:25.042915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 566/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:24 2022] GET /identity/v3/users/d397240bd1774309a092ecee3f000184 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:25.048743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856290c-3ef1-4e5b-a320-17ab1f90e1ac None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:25.107281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:25.147707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856290c-3ef1-4e5b-a320-17ab1f90e1ac None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:25.180173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fa98f-1c61-4ebe-a938-6a9582ca3465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:25.180804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fa98f-1c61-4ebe-a938-6a9582ca3465 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:25.181086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fa98f-1c61-4ebe-a938-6a9582ca3465 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:25.181362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fa98f-1c61-4ebe-a938-6a9582ca3465 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:25.245306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fa98f-1c61-4ebe-a938-6a9582ca3465 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:25.246788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fa98f-1c61-4ebe-a938-6a9582ca3465 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:25.300647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:25.342286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 562/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:52:25.348459 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4430d0d-5676-4203-a050-c176302002a8 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:25.390518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:25.445764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4430d0d-5676-4203-a050-c176302002a8 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:25.477775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d0884-df53-4f75-847a-eb065c9f2d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:25.478640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0884-df53-4f75-847a-eb065c9f2d65 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:25.478922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0884-df53-4f75-847a-eb065c9f2d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:25.479193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0884-df53-4f75-847a-eb065c9f2d65 None admin] PATH_INFO: `/v3/projects/9c1c61d756484d07b712e8557a541cd1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:25.480341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d0884-df53-4f75-847a-eb065c9f2d65 None admin] RBAC: Authorizing `identity:get_project(project_id=9c1c61d756484d07b712e8557a541cd1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:25.481753 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d0884-df53-4f75-847a-eb065c9f2d65 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:25.483450 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 567/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:25 2022] GET /identity/v3/projects/9c1c61d756484d07b712e8557a541cd1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:25.489581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fa98f-1c61-4ebe-a938-6a9582ca3465 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:25.585852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fa98f-1c61-4ebe-a938-6a9582ca3465 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:25.618142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1424ad-5ee1-423d-bf27-b6dec4d1225b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:25.618806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1424ad-5ee1-423d-bf27-b6dec4d1225b None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:25.619094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1424ad-5ee1-423d-bf27-b6dec4d1225b None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:25.619374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1424ad-5ee1-423d-bf27-b6dec4d1225b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:25.620195 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1424ad-5ee1-423d-bf27-b6dec4d1225b None admin] RBAC: Adding query filter params (role.id=fa67cb2f211d433097f606ff029f863e, scope.project.id=9c1c61d756484d07b712e8557a541cd1, user.id=d397240bd1774309a092ecee3f000184) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:25.620511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1424ad-5ee1-423d-bf27-b6dec4d1225b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:25.621943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1424ad-5ee1-423d-bf27-b6dec4d1225b None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:25.627581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 563/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:52:25 2022] GET /identity/v3/role_assignments?user.id=d397240bd1774309a092ecee3f000184&scope.project.id=9c1c61d756484d07b712e8557a541cd1&role.id=fa67cb2f211d433097f606ff029f863e => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:27.794003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bee12-3c1d-45be-b1f4-f634c47a1574 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:27.794555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bee12-3c1d-45be-b1f4-f634c47a1574 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:27.794636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bee12-3c1d-45be-b1f4-f634c47a1574 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:27.795950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 568/1131] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:27.801013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb05dcc-be0d-42a1-b493-408f378640ce None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:27.801410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb05dcc-be0d-42a1-b493-408f378640ce None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:27.801557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb05dcc-be0d-42a1-b493-408f378640ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:27.818333 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fcb05dcc-be0d-42a1-b493-408f378640ce None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:27.819526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fcb05dcc-be0d-42a1-b493-408f378640ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:27.835569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:27.882885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 564/1132] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:27.905988 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bee12-3c1d-45be-b1f4-f634c47a1574 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:28.003348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bee12-3c1d-45be-b1f4-f634c47a1574 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:28.012423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:28.037407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d06c1-e6ff-4e98-a915-81af4062355d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:28.038182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d06c1-e6ff-4e98-a915-81af4062355d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:28.038485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d06c1-e6ff-4e98-a915-81af4062355d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:28.038766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d06c1-e6ff-4e98-a915-81af4062355d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:28.039928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d06c1-e6ff-4e98-a915-81af4062355d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:28.041348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d06c1-e6ff-4e98-a915-81af4062355d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:28.042975 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 569/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:52:28.048564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb05dcc-be0d-42a1-b493-408f378640ce None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:28.143025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb05dcc-be0d-42a1-b493-408f378640ce None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:28.175672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c47bc-b58b-496d-b71f-abe072a52784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:28.176325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c47bc-b58b-496d-b71f-abe072a52784 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:28.176637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c47bc-b58b-496d-b71f-abe072a52784 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:28.176914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c47bc-b58b-496d-b71f-abe072a52784 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:28.177612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c47bc-b58b-496d-b71f-abe072a52784 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:28.179073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c47bc-b58b-496d-b71f-abe072a52784 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:28.203177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 565/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:52:28 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:52:30.408275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352067e-10cc-46cf-a7a0-50afe3764df7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:30.408791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352067e-10cc-46cf-a7a0-50afe3764df7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:30.408883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352067e-10cc-46cf-a7a0-50afe3764df7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:30.410189 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 570/1135] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:30.416079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699b57-10c9-4ab3-b815-871807ca528e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:30.416480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699b57-10c9-4ab3-b815-871807ca528e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:30.416644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699b57-10c9-4ab3-b815-871807ca528e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:30.422590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:30.433733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2e699b57-10c9-4ab3-b815-871807ca528e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:30.434963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2e699b57-10c9-4ab3-b815-871807ca528e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:30.498283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 566/1136] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:30.523624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352067e-10cc-46cf-a7a0-50afe3764df7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:30.621038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352067e-10cc-46cf-a7a0-50afe3764df7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:30.653965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21acb13-2b52-487c-98e7-e742e57d1567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:30.654789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21acb13-2b52-487c-98e7-e742e57d1567 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:30.655071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21acb13-2b52-487c-98e7-e742e57d1567 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:30.655345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21acb13-2b52-487c-98e7-e742e57d1567 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:30.656494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21acb13-2b52-487c-98e7-e742e57d1567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:30.657917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21acb13-2b52-487c-98e7-e742e57d1567 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:30.659595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 571/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:52:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:30.665283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e699b57-10c9-4ab3-b815-871807ca528e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:30.761235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e699b57-10c9-4ab3-b815-871807ca528e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:30.794246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cbbf34-b5a4-438a-80b8-a78752a3cfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:30.794929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbbf34-b5a4-438a-80b8-a78752a3cfc4 None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:30.795218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbbf34-b5a4-438a-80b8-a78752a3cfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:30.795499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cbbf34-b5a4-438a-80b8-a78752a3cfc4 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:30.796197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbbf34-b5a4-438a-80b8-a78752a3cfc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:30.797634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cbbf34-b5a4-438a-80b8-a78752a3cfc4 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:30.804756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:30.819095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 567/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:52:30 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:52:32.996797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766af9ed-018a-4ad5-9abb-42caccf51aad None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:32.997305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766af9ed-018a-4ad5-9abb-42caccf51aad None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:32.997497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766af9ed-018a-4ad5-9abb-42caccf51aad None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:32.998740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 572/1139] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:33.004039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82ef6f-d162-4df8-8040-6db349e65176 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:33.004437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82ef6f-d162-4df8-8040-6db349e65176 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:33.004638 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82ef6f-d162-4df8-8040-6db349e65176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:33.021372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7d82ef6f-d162-4df8-8040-6db349e65176 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:33.022581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7d82ef6f-d162-4df8-8040-6db349e65176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:33.087694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 568/1140] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 20:52:33.112421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766af9ed-018a-4ad5-9abb-42caccf51aad None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:33.158010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:33.210882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766af9ed-018a-4ad5-9abb-42caccf51aad None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:33.243369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb661b73-8e41-48e9-a774-8f82601e7cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:33.244147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb661b73-8e41-48e9-a774-8f82601e7cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:33.244433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb661b73-8e41-48e9-a774-8f82601e7cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:33.244712 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb661b73-8e41-48e9-a774-8f82601e7cb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:33.254484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb661b73-8e41-48e9-a774-8f82601e7cb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-bb661b73-8e41-48e9-a774-8f82601e7cb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:33.262376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:33.263508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:33.264597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 573/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:52:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:33.269532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82ef6f-d162-4df8-8040-6db349e65176 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:33.364722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82ef6f-d162-4df8-8040-6db349e65176 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:33.383652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:33.399571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06080fc-7367-4b2d-b120-c677f95e77cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:33.400239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06080fc-7367-4b2d-b120-c677f95e77cb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:33.400521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06080fc-7367-4b2d-b120-c677f95e77cb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:33.400798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06080fc-7367-4b2d-b120-c677f95e77cb None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:33.401549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06080fc-7367-4b2d-b120-c677f95e77cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:33.401853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06080fc-7367-4b2d-b120-c677f95e77cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:33.403267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06080fc-7367-4b2d-b120-c677f95e77cb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:33.411933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 569/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:52:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:33.417710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb661b73-8e41-48e9-a774-8f82601e7cb6 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:33.514066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb661b73-8e41-48e9-a774-8f82601e7cb6 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:33.516333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:33.546611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc946d2-0e37-4eb0-a7f1-29846fe5ae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:33.547461 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc946d2-0e37-4eb0-a7f1-29846fe5ae4f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:33.547746 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc946d2-0e37-4eb0-a7f1-29846fe5ae4f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:33.548021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc946d2-0e37-4eb0-a7f1-29846fe5ae4f None admin] PATH_INFO: `/v3/users/67051b4d82ed4e0d8f8f691f4d822e96` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:33.561005 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc946d2-0e37-4eb0-a7f1-29846fe5ae4f None admin] RBAC: Authorizing `identity:get_user(user_id=67051b4d82ed4e0d8f8f691f4d822e96)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:33.563235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc946d2-0e37-4eb0-a7f1-29846fe5ae4f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:33.565014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 574/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:33 2022] GET /identity/v3/users/67051b4d82ed4e0d8f8f691f4d822e96 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:33.571497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06080fc-7367-4b2d-b120-c677f95e77cb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:33.666886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06080fc-7367-4b2d-b120-c677f95e77cb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:33.698658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd9a24-8bc9-4333-8fa4-64c20294dce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:33.699289 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd9a24-8bc9-4333-8fa4-64c20294dce2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:33.699599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd9a24-8bc9-4333-8fa4-64c20294dce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:33.699876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd9a24-8bc9-4333-8fa4-64c20294dce2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:33.746277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:33.764750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd9a24-8bc9-4333-8fa4-64c20294dce2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:33.766197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd9a24-8bc9-4333-8fa4-64c20294dce2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:33.859728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 570/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 20:52:33.866007 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc946d2-0e37-4eb0-a7f1-29846fe5ae4f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:33.960904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc946d2-0e37-4eb0-a7f1-29846fe5ae4f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:33.993168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e55de6-a26d-41c8-b994-a425f8e974a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:33.993999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e55de6-a26d-41c8-b994-a425f8e974a2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:33.994281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e55de6-a26d-41c8-b994-a425f8e974a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:33.994586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e55de6-a26d-41c8-b994-a425f8e974a2 None admin] PATH_INFO: `/v3/projects/9ac23940c70440dfa9cd6c6b9b238f0a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:33.995732 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e55de6-a26d-41c8-b994-a425f8e974a2 None admin] RBAC: Authorizing `identity:get_project(project_id=9ac23940c70440dfa9cd6c6b9b238f0a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:33.997164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e55de6-a26d-41c8-b994-a425f8e974a2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:33.998843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 575/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:33 2022] GET /identity/v3/projects/9ac23940c70440dfa9cd6c6b9b238f0a => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:34.004826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd9a24-8bc9-4333-8fa4-64c20294dce2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:34.081453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:34.099299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd9a24-8bc9-4333-8fa4-64c20294dce2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:34.132753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf44046-2ec3-4090-b4fe-021453fb936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:34.133388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf44046-2ec3-4090-b4fe-021453fb936a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:34.133670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf44046-2ec3-4090-b4fe-021453fb936a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:34.134210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf44046-2ec3-4090-b4fe-021453fb936a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:34.135058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf44046-2ec3-4090-b4fe-021453fb936a None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=9ac23940c70440dfa9cd6c6b9b238f0a, user.id=67051b4d82ed4e0d8f8f691f4d822e96) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:34.135369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf44046-2ec3-4090-b4fe-021453fb936a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:34.136794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf44046-2ec3-4090-b4fe-021453fb936a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:34.141980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 571/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:52:34 2022] GET /identity/v3/role_assignments?user.id=67051b4d82ed4e0d8f8f691f4d822e96&scope.project.id=9ac23940c70440dfa9cd6c6b9b238f0a&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:36.294368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985f3a1-5578-4f16-afc6-5f7f67fa0d47 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:36.294915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985f3a1-5578-4f16-afc6-5f7f67fa0d47 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:36.295010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985f3a1-5578-4f16-afc6-5f7f67fa0d47 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:36.296315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 576/1147] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:36.302145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba0993-3eaf-417b-9201-cbd571dfa3e6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:36.302588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba0993-3eaf-417b-9201-cbd571dfa3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:36.302737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba0993-3eaf-417b-9201-cbd571dfa3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:36.319687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-14ba0993-3eaf-417b-9201-cbd571dfa3e6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:36.320855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-14ba0993-3eaf-417b-9201-cbd571dfa3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:36.384738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 572/1148] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:36.406800 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985f3a1-5578-4f16-afc6-5f7f67fa0d47 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:36.444262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:36.513596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985f3a1-5578-4f16-afc6-5f7f67fa0d47 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:36.545795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb10a2-2825-4025-9a17-6539fa46605f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:36.546545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb10a2-2825-4025-9a17-6539fa46605f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:36.546833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb10a2-2825-4025-9a17-6539fa46605f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:36.547110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb10a2-2825-4025-9a17-6539fa46605f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:36.557399 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb10a2-2825-4025-9a17-6539fa46605f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-65fb10a2-2825-4025-9a17-6539fa46605f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:36.565223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:36.566344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:36.567697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 577/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:52:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:36.572503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba0993-3eaf-417b-9201-cbd571dfa3e6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:36.644622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:36.666962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba0993-3eaf-417b-9201-cbd571dfa3e6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:36.703474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d855ae-f442-4058-8128-de50c8a5fa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:36.704317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d855ae-f442-4058-8128-de50c8a5fa88 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:36.704599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d855ae-f442-4058-8128-de50c8a5fa88 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:36.704873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d855ae-f442-4058-8128-de50c8a5fa88 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:36.705615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d855ae-f442-4058-8128-de50c8a5fa88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:36.705922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d855ae-f442-4058-8128-de50c8a5fa88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:36.707530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d855ae-f442-4058-8128-de50c8a5fa88 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:36.721669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 573/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:52:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:36.728128 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb10a2-2825-4025-9a17-6539fa46605f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:36.816867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:36.826968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb10a2-2825-4025-9a17-6539fa46605f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:36.860152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c887a-282b-49f2-b76e-27aefadc8440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:36.861008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c887a-282b-49f2-b76e-27aefadc8440 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:36.861296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c887a-282b-49f2-b76e-27aefadc8440 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:36.861575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c887a-282b-49f2-b76e-27aefadc8440 None admin] PATH_INFO: `/v3/users/67051b4d82ed4e0d8f8f691f4d822e96` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:36.862787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c887a-282b-49f2-b76e-27aefadc8440 None admin] RBAC: Authorizing `identity:get_user(user_id=67051b4d82ed4e0d8f8f691f4d822e96)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:36.864254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81c887a-282b-49f2-b76e-27aefadc8440 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:36.865879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 578/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:36 2022] GET /identity/v3/users/67051b4d82ed4e0d8f8f691f4d822e96 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:36.871610 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d855ae-f442-4058-8128-de50c8a5fa88 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:36.966532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d855ae-f442-4058-8128-de50c8a5fa88 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:36.998305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de633fd6-3270-4f12-9511-f662e9a0178a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:36.999174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de633fd6-3270-4f12-9511-f662e9a0178a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:36.999457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de633fd6-3270-4f12-9511-f662e9a0178a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:36.999731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de633fd6-3270-4f12-9511-f662e9a0178a None admin] PATH_INFO: `/v3/projects/9ac23940c70440dfa9cd6c6b9b238f0a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:37.000900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de633fd6-3270-4f12-9511-f662e9a0178a None admin] RBAC: Authorizing `identity:get_project(project_id=9ac23940c70440dfa9cd6c6b9b238f0a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:37.002328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de633fd6-3270-4f12-9511-f662e9a0178a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:37.004002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 574/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:36 2022] GET /identity/v3/projects/9ac23940c70440dfa9cd6c6b9b238f0a => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:37.009994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c887a-282b-49f2-b76e-27aefadc8440 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:37.112869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c887a-282b-49f2-b76e-27aefadc8440 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:37.145508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f06a62f-aec9-49ab-bda0-e569a784e17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:37.146432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06a62f-aec9-49ab-bda0-e569a784e17c None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:37.146875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06a62f-aec9-49ab-bda0-e569a784e17c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:37.147153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f06a62f-aec9-49ab-bda0-e569a784e17c None admin] PATH_INFO: `/v3/projects/9ac23940c70440dfa9cd6c6b9b238f0a/users/67051b4d82ed4e0d8f8f691f4d822e96/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:37.149092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06a62f-aec9-49ab-bda0-e569a784e17c None admin] RBAC: Authorizing `identity:create_grant(project_id=9ac23940c70440dfa9cd6c6b9b238f0a, user_id=67051b4d82ed4e0d8f8f691f4d822e96, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:37.150584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f06a62f-aec9-49ab-bda0-e569a784e17c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:37.164749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 579/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:52:37 2022] PUT /identity/v3/projects/9ac23940c70440dfa9cd6c6b9b238f0a/users/67051b4d82ed4e0d8f8f691f4d822e96/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:52:39.314960 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9182a-b7eb-4566-8f10-9849199179ed None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:39.315482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9182a-b7eb-4566-8f10-9849199179ed None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:39.315576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9182a-b7eb-4566-8f10-9849199179ed None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:39.317470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 575/1154] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:39.322376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476d583-424a-4e0c-9a73-3bec0fce4c76 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:39.322816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476d583-424a-4e0c-9a73-3bec0fce4c76 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:39.322973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9476d583-424a-4e0c-9a73-3bec0fce4c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:39.339314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9476d583-424a-4e0c-9a73-3bec0fce4c76 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:39.340520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9476d583-424a-4e0c-9a73-3bec0fce4c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:39.353342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:39.406287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 580/1155] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:39.429457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9182a-b7eb-4566-8f10-9849199179ed None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:39.438537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:39.526793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9182a-b7eb-4566-8f10-9849199179ed None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:39.559132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9ccff-84ef-4bae-ab5e-56a6d166e5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:39.559926 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9ccff-84ef-4bae-ab5e-56a6d166e5ce None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:39.560209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9ccff-84ef-4bae-ab5e-56a6d166e5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:39.560481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9ccff-84ef-4bae-ab5e-56a6d166e5ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:39.570302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9ccff-84ef-4bae-ab5e-56a6d166e5ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-32a9ccff-84ef-4bae-ab5e-56a6d166e5ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:39.577898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:39.579077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:39.580003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 576/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:52:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:39.584929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476d583-424a-4e0c-9a73-3bec0fce4c76 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:39.680490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9476d583-424a-4e0c-9a73-3bec0fce4c76 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:39.712088 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4485338-bdf8-42ab-bf2e-91be74b11344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:39.712747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4485338-bdf8-42ab-bf2e-91be74b11344 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:39.713133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4485338-bdf8-42ab-bf2e-91be74b11344 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:39.713454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4485338-bdf8-42ab-bf2e-91be74b11344 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:39.714148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4485338-bdf8-42ab-bf2e-91be74b11344 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:39.714479 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4485338-bdf8-42ab-bf2e-91be74b11344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:39.715852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4485338-bdf8-42ab-bf2e-91be74b11344 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:39.724239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 581/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:52:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:39.729863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9ccff-84ef-4bae-ab5e-56a6d166e5ce None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:39.793198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:39.827890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9ccff-84ef-4bae-ab5e-56a6d166e5ce None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:39.860436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530739c4-846a-4a7a-89f8-250c7afc0183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:39.861291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530739c4-846a-4a7a-89f8-250c7afc0183 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:39.861577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530739c4-846a-4a7a-89f8-250c7afc0183 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:39.861857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530739c4-846a-4a7a-89f8-250c7afc0183 None admin] PATH_INFO: `/v3/users/67051b4d82ed4e0d8f8f691f4d822e96` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:39.863054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530739c4-846a-4a7a-89f8-250c7afc0183 None admin] RBAC: Authorizing `identity:get_user(user_id=67051b4d82ed4e0d8f8f691f4d822e96)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:39.864528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530739c4-846a-4a7a-89f8-250c7afc0183 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:39.866136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 577/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:39 2022] GET /identity/v3/users/67051b4d82ed4e0d8f8f691f4d822e96 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:39.871945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4485338-bdf8-42ab-bf2e-91be74b11344 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:39.884539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:39.968643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4485338-bdf8-42ab-bf2e-91be74b11344 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:40.000998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe2cff-3e8b-4929-a507-e4df1ba2def0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:40.001635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe2cff-3e8b-4929-a507-e4df1ba2def0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:40.001921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe2cff-3e8b-4929-a507-e4df1ba2def0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:40.002202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe2cff-3e8b-4929-a507-e4df1ba2def0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:40.067918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe2cff-3e8b-4929-a507-e4df1ba2def0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:40.070363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe2cff-3e8b-4929-a507-e4df1ba2def0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:40.084073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:40.168586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 582/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:52:40.174511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530739c4-846a-4a7a-89f8-250c7afc0183 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:40.271922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530739c4-846a-4a7a-89f8-250c7afc0183 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:40.303518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e1deb-ff08-469e-9b6a-5e545b138e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:40.304366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e1deb-ff08-469e-9b6a-5e545b138e3a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:40.304653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e1deb-ff08-469e-9b6a-5e545b138e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:40.304934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e1deb-ff08-469e-9b6a-5e545b138e3a None admin] PATH_INFO: `/v3/projects/9ac23940c70440dfa9cd6c6b9b238f0a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:40.306095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e1deb-ff08-469e-9b6a-5e545b138e3a None admin] RBAC: Authorizing `identity:get_project(project_id=9ac23940c70440dfa9cd6c6b9b238f0a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:40.307570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e1deb-ff08-469e-9b6a-5e545b138e3a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:40.309223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 578/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:40 2022] GET /identity/v3/projects/9ac23940c70440dfa9cd6c6b9b238f0a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:40.315085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe2cff-3e8b-4929-a507-e4df1ba2def0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:40.428596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe2cff-3e8b-4929-a507-e4df1ba2def0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:40.438718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:40.462717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17eac93-f3ee-4387-9208-bb366146465c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:40.463360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eac93-f3ee-4387-9208-bb366146465c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:40.463643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eac93-f3ee-4387-9208-bb366146465c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:40.463916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eac93-f3ee-4387-9208-bb366146465c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:40.464724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17eac93-f3ee-4387-9208-bb366146465c None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=9ac23940c70440dfa9cd6c6b9b238f0a, user.id=67051b4d82ed4e0d8f8f691f4d822e96) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:40.465031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17eac93-f3ee-4387-9208-bb366146465c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:40.466476 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17eac93-f3ee-4387-9208-bb366146465c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:40.472093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 583/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:52:40 2022] GET /identity/v3/role_assignments?user.id=67051b4d82ed4e0d8f8f691f4d822e96&scope.project.id=9ac23940c70440dfa9cd6c6b9b238f0a&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:42.673303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c334a1-256c-4c8b-afd7-64c6215334f3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:42.673849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c334a1-256c-4c8b-afd7-64c6215334f3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:42.673849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c334a1-256c-4c8b-afd7-64c6215334f3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:42.675247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 579/1162] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:42.680527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cfdb1-4960-4367-849d-a700609ae33e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:42.680912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cfdb1-4960-4367-849d-a700609ae33e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:42.681083 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613cfdb1-4960-4367-849d-a700609ae33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:42.697834 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-613cfdb1-4960-4367-849d-a700609ae33e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:42.699042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-613cfdb1-4960-4367-849d-a700609ae33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:42.762605 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 584/1163] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:42.784724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c334a1-256c-4c8b-afd7-64c6215334f3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:42.787558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:42.884850 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c334a1-256c-4c8b-afd7-64c6215334f3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:42.916146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa77e89-c7e6-427f-b844-cf1c9f7e1c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:42.916782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa77e89-c7e6-427f-b844-cf1c9f7e1c03 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:42.917067 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa77e89-c7e6-427f-b844-cf1c9f7e1c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:42.917345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa77e89-c7e6-427f-b844-cf1c9f7e1c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:42.980819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa77e89-c7e6-427f-b844-cf1c9f7e1c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:42.982211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa77e89-c7e6-427f-b844-cf1c9f7e1c03 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:42.983959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:43.076419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 580/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:52:43.082369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cfdb1-4960-4367-849d-a700609ae33e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:43.175386 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613cfdb1-4960-4367-849d-a700609ae33e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:43.178099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:43.209558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14f9a9-56ea-4b3d-97f9-e4bead65272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:43.210350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14f9a9-56ea-4b3d-97f9-e4bead65272c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:43.210701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14f9a9-56ea-4b3d-97f9-e4bead65272c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:43.210977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14f9a9-56ea-4b3d-97f9-e4bead65272c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:43.217380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14f9a9-56ea-4b3d-97f9-e4bead65272c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:43.218846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d14f9a9-56ea-4b3d-97f9-e4bead65272c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1d14f9a9-56ea-4b3d-97f9-e4bead65272c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:43.232999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:43.239162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:43.240275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 585/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 26 20:52:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:43.241326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa77e89-c7e6-427f-b844-cf1c9f7e1c03 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:43.335392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa77e89-c7e6-427f-b844-cf1c9f7e1c03 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:43.336863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:43.369248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c94a1-f3b9-4795-95e5-cc089fb5b075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:43.369896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c94a1-f3b9-4795-95e5-cc089fb5b075 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:43.370181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c94a1-f3b9-4795-95e5-cc089fb5b075 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:43.370475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c94a1-f3b9-4795-95e5-cc089fb5b075 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:43.371191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c94a1-f3b9-4795-95e5-cc089fb5b075 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:43.371497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c94a1-f3b9-4795-95e5-cc089fb5b075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:43.372871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c94a1-f3b9-4795-95e5-cc089fb5b075 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:43.379807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 581/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Thu May 26 20:52:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:45.525792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b221f15-d919-4f4d-a090-13eda76cdf34 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:45.526381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b221f15-d919-4f4d-a090-13eda76cdf34 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:45.526381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b221f15-d919-4f4d-a090-13eda76cdf34 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:45.527729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 586/1167] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:45.532801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b524644-ca80-4369-94da-424341a3f62f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:45.533241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b524644-ca80-4369-94da-424341a3f62f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:45.533368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b524644-ca80-4369-94da-424341a3f62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:45.549749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3b524644-ca80-4369-94da-424341a3f62f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:45.550964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3b524644-ca80-4369-94da-424341a3f62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:45.614884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 582/1168] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:45.637666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b221f15-d919-4f4d-a090-13eda76cdf34 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:45.734322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b221f15-d919-4f4d-a090-13eda76cdf34 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:45.766073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c94297-b72d-460c-864c-3645b4fb8c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:45.766888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c94297-b72d-460c-864c-3645b4fb8c1b None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:45.767173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c94297-b72d-460c-864c-3645b4fb8c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:45.767450 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c94297-b72d-460c-864c-3645b4fb8c1b None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:45.768172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c94297-b72d-460c-864c-3645b4fb8c1b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:45.769529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c94297-b72d-460c-864c-3645b4fb8c1b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:45.783998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 587/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Thu May 26 20:52:45 2022] POST /identity/v3/domains => generated 267 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:52:47.946056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58573998-5370-44f9-b942-9b0227bb1747 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:47.946594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58573998-5370-44f9-b942-9b0227bb1747 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:47.946707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58573998-5370-44f9-b942-9b0227bb1747 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:47.948013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 583/1170] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:47.953240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d207bb-9647-46e0-9ffb-3d79c9ece03d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:47.953648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d207bb-9647-46e0-9ffb-3d79c9ece03d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:47.953796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d207bb-9647-46e0-9ffb-3d79c9ece03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:47.970122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d7d207bb-9647-46e0-9ffb-3d79c9ece03d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:47.971340 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d7d207bb-9647-46e0-9ffb-3d79c9ece03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:47.989003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:48.037995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 588/1171] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:48.061104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58573998-5370-44f9-b942-9b0227bb1747 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:48.152531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:48.159335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58573998-5370-44f9-b942-9b0227bb1747 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:48.191776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53682230-ff67-47e1-90a6-a7d182ab500d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:48.192650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53682230-ff67-47e1-90a6-a7d182ab500d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:48.192940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53682230-ff67-47e1-90a6-a7d182ab500d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:48.193222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53682230-ff67-47e1-90a6-a7d182ab500d None admin] PATH_INFO: `/v3/domains/d600eaf705154314980ee4ee448da031` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:48.203311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53682230-ff67-47e1-90a6-a7d182ab500d None admin] RBAC: Authorizing `identity:get_domain(domain_id=d600eaf705154314980ee4ee448da031)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:48.204763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53682230-ff67-47e1-90a6-a7d182ab500d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:48.206380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 584/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 26 20:52:48 2022] GET /identity/v3/domains/d600eaf705154314980ee4ee448da031 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:48.212664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d207bb-9647-46e0-9ffb-3d79c9ece03d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:48.307801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d207bb-9647-46e0-9ffb-3d79c9ece03d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:48.339828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a37969-2d67-4c0d-a851-81c3b758dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:48.340469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a37969-2d67-4c0d-a851-81c3b758dedc None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:48.340750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a37969-2d67-4c0d-a851-81c3b758dedc None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:48.341024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a37969-2d67-4c0d-a851-81c3b758dedc None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:48.341725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a37969-2d67-4c0d-a851-81c3b758dedc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:48.343163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a37969-2d67-4c0d-a851-81c3b758dedc None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:48.366909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 589/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:52:48 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:52:50.588687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db08f3-04b1-420b-b98f-791536d1b28f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:50.589258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db08f3-04b1-420b-b98f-791536d1b28f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:50.589258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db08f3-04b1-420b-b98f-791536d1b28f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:50.590618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 585/1174] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:50.595826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad305a-ad56-400e-92fd-102a82befbfe None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:50.596213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad305a-ad56-400e-92fd-102a82befbfe None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:50.596389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad305a-ad56-400e-92fd-102a82befbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:50.608480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:50.614200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-63ad305a-ad56-400e-92fd-102a82befbfe None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:50.615411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-63ad305a-ad56-400e-92fd-102a82befbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:50.679809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 590/1175] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:50.704554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db08f3-04b1-420b-b98f-791536d1b28f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:50.801789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db08f3-04b1-420b-b98f-791536d1b28f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:50.835829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28957cc-b12b-4b59-8308-efa9522b62b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:50.836674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28957cc-b12b-4b59-8308-efa9522b62b1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:50.836955 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28957cc-b12b-4b59-8308-efa9522b62b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:50.837226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28957cc-b12b-4b59-8308-efa9522b62b1 None admin] PATH_INFO: `/v3/domains/d600eaf705154314980ee4ee448da031` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:50.838385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28957cc-b12b-4b59-8308-efa9522b62b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d600eaf705154314980ee4ee448da031)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:50.839896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28957cc-b12b-4b59-8308-efa9522b62b1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:50.841427 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 586/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Thu May 26 20:52:50 2022] GET /identity/v3/domains/d600eaf705154314980ee4ee448da031 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:50.847117 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad305a-ad56-400e-92fd-102a82befbfe None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:50.940354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad305a-ad56-400e-92fd-102a82befbfe None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:50.973262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf6fc7-95f6-4612-ab55-bdd71d48f2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:50.973916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf6fc7-95f6-4612-ab55-bdd71d48f2c2 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:50.974201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf6fc7-95f6-4612-ab55-bdd71d48f2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:50.974515 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf6fc7-95f6-4612-ab55-bdd71d48f2c2 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:50.975213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf6fc7-95f6-4612-ab55-bdd71d48f2c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:50.976627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf6fc7-95f6-4612-ab55-bdd71d48f2c2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:50.983949 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:50.996659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 591/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:52:50 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:52:53.201657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b1739-2d91-41fd-9bb4-85949a256797 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:53.202165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b1739-2d91-41fd-9bb4-85949a256797 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:53.202165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b1739-2d91-41fd-9bb4-85949a256797 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:53.203516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 587/1178] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:53.209313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50460fad-ea8c-4930-9fb2-fbf373821785 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:53.209756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50460fad-ea8c-4930-9fb2-fbf373821785 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:53.209860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50460fad-ea8c-4930-9fb2-fbf373821785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:53.226125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-50460fad-ea8c-4930-9fb2-fbf373821785 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:53.227324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-50460fad-ea8c-4930-9fb2-fbf373821785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:53.291958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 592/1179] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:53.315170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b1739-2d91-41fd-9bb4-85949a256797 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:53.339833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:53.413438 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b1739-2d91-41fd-9bb4-85949a256797 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:53.449636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98e1c5-51b8-485d-bf75-2894372c135c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:53.450409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98e1c5-51b8-485d-bf75-2894372c135c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:53.450722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98e1c5-51b8-485d-bf75-2894372c135c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:53.451000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98e1c5-51b8-485d-bf75-2894372c135c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:53.460796 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98e1c5-51b8-485d-bf75-2894372c135c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-bf98e1c5-51b8-485d-bf75-2894372c135c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:53.468298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:53.469472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:53.470475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 588/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:52:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:53.475455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50460fad-ea8c-4930-9fb2-fbf373821785 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:53.570849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50460fad-ea8c-4930-9fb2-fbf373821785 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:53.592229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:53.604108 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b79d4-024b-450b-bac0-66073c8f01b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:53.604762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b79d4-024b-450b-bac0-66073c8f01b2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:53.605102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b79d4-024b-450b-bac0-66073c8f01b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:53.605383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b79d4-024b-450b-bac0-66073c8f01b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:53.606117 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b79d4-024b-450b-bac0-66073c8f01b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:53.606466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b79d4-024b-450b-bac0-66073c8f01b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:53.607865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b79d4-024b-450b-bac0-66073c8f01b2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:53.616150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 593/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:52:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:53.621807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98e1c5-51b8-485d-bf75-2894372c135c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:53.705619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:53.724867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98e1c5-51b8-485d-bf75-2894372c135c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:53.757880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c74040-807b-4db5-b043-e2241936aa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:53.759037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c74040-807b-4db5-b043-e2241936aa47 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:53.759322 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c74040-807b-4db5-b043-e2241936aa47 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:53.759600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c74040-807b-4db5-b043-e2241936aa47 None admin] PATH_INFO: `/v3/users/48aae7da46d44547b65e5c01dd7fd740` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:53.772786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c74040-807b-4db5-b043-e2241936aa47 None admin] RBAC: Authorizing `identity:get_user(user_id=48aae7da46d44547b65e5c01dd7fd740)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:53.774239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c74040-807b-4db5-b043-e2241936aa47 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:53.775907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 589/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:53 2022] GET /identity/v3/users/48aae7da46d44547b65e5c01dd7fd740 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:53.781643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b79d4-024b-450b-bac0-66073c8f01b2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:53.879387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b79d4-024b-450b-bac0-66073c8f01b2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:53.911569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22460b8d-95be-48d0-96a8-5766c90a3a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:53.912204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22460b8d-95be-48d0-96a8-5766c90a3a2a None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:53.912484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22460b8d-95be-48d0-96a8-5766c90a3a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:53.912761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22460b8d-95be-48d0-96a8-5766c90a3a2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:53.961737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:53.977504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22460b8d-95be-48d0-96a8-5766c90a3a2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:53.978935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22460b8d-95be-48d0-96a8-5766c90a3a2a None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:54.082437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 594/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:52:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:52:54.088347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c74040-807b-4db5-b043-e2241936aa47 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:54.183648 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c74040-807b-4db5-b043-e2241936aa47 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:54.216151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbbbda-46ef-4ee2-9608-ca338a26229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:54.216992 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbbbda-46ef-4ee2-9608-ca338a26229e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:54.217270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbbbda-46ef-4ee2-9608-ca338a26229e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:54.217543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbbbda-46ef-4ee2-9608-ca338a26229e None admin] PATH_INFO: `/v3/projects/6696d1c87c26481d8def5b6090b6bdfb` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:54.218721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfbbbda-46ef-4ee2-9608-ca338a26229e None admin] RBAC: Authorizing `identity:get_project(project_id=6696d1c87c26481d8def5b6090b6bdfb)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:54.220162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfbbbda-46ef-4ee2-9608-ca338a26229e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:54.221795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 590/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:54 2022] GET /identity/v3/projects/6696d1c87c26481d8def5b6090b6bdfb => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:54.227652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22460b8d-95be-48d0-96a8-5766c90a3a2a None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:54.307815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:54.323728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22460b8d-95be-48d0-96a8-5766c90a3a2a None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:54.361547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae204caa-4a0c-49bf-a03f-2ea1a72fba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:54.362197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae204caa-4a0c-49bf-a03f-2ea1a72fba4b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:54.362506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae204caa-4a0c-49bf-a03f-2ea1a72fba4b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:54.362787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae204caa-4a0c-49bf-a03f-2ea1a72fba4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:54.364109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae204caa-4a0c-49bf-a03f-2ea1a72fba4b None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=6696d1c87c26481d8def5b6090b6bdfb, user.id=48aae7da46d44547b65e5c01dd7fd740) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:54.364423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae204caa-4a0c-49bf-a03f-2ea1a72fba4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:54.365844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae204caa-4a0c-49bf-a03f-2ea1a72fba4b None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:54.371190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 595/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:52:54 2022] GET /identity/v3/role_assignments?user.id=48aae7da46d44547b65e5c01dd7fd740&scope.project.id=6696d1c87c26481d8def5b6090b6bdfb&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:56.523761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e87fd-327d-40ee-b42b-0d412bdc1481 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:56.524314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e87fd-327d-40ee-b42b-0d412bdc1481 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:56.524314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e87fd-327d-40ee-b42b-0d412bdc1481 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:56.525653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 591/1186] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:56.531074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329286-5846-4f7f-af4c-c8b421e64f14 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:56.531447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329286-5846-4f7f-af4c-c8b421e64f14 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:56.531633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02329286-5846-4f7f-af4c-c8b421e64f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:56.548650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-02329286-5846-4f7f-af4c-c8b421e64f14 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:56.549822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-02329286-5846-4f7f-af4c-c8b421e64f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:56.613535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 596/1187] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:56.635620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e87fd-327d-40ee-b42b-0d412bdc1481 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:56.648102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:56.733601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e87fd-327d-40ee-b42b-0d412bdc1481 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:56.765932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca07ae-8359-47b8-9372-cfacbc243d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:56.766727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca07ae-8359-47b8-9372-cfacbc243d3c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:56.767012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca07ae-8359-47b8-9372-cfacbc243d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:56.767289 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca07ae-8359-47b8-9372-cfacbc243d3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:56.777633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca07ae-8359-47b8-9372-cfacbc243d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-57ca07ae-8359-47b8-9372-cfacbc243d3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:56.785235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:56.786341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:52:56.787606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 592/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:52:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:56.792275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02329286-5846-4f7f-af4c-c8b421e64f14 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:56.866616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:56.887747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02329286-5846-4f7f-af4c-c8b421e64f14 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:56.919879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94b261-eead-410c-bf6f-18ca074f1b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:56.920529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94b261-eead-410c-bf6f-18ca074f1b87 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:56.920809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94b261-eead-410c-bf6f-18ca074f1b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:56.921083 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94b261-eead-410c-bf6f-18ca074f1b87 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:56.921806 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94b261-eead-410c-bf6f-18ca074f1b87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:56.922113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94b261-eead-410c-bf6f-18ca074f1b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:56.923510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94b261-eead-410c-bf6f-18ca074f1b87 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:56.931251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 597/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:52:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:56.937049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca07ae-8359-47b8-9372-cfacbc243d3c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:57.004537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:57.035836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca07ae-8359-47b8-9372-cfacbc243d3c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:57.068461 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08833c1-dccb-4588-bc74-b5c6e58d4c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:57.069306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08833c1-dccb-4588-bc74-b5c6e58d4c29 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:57.069593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08833c1-dccb-4588-bc74-b5c6e58d4c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:57.069874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08833c1-dccb-4588-bc74-b5c6e58d4c29 None admin] PATH_INFO: `/v3/users/48aae7da46d44547b65e5c01dd7fd740` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:57.071071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08833c1-dccb-4588-bc74-b5c6e58d4c29 None admin] RBAC: Authorizing `identity:get_user(user_id=48aae7da46d44547b65e5c01dd7fd740)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:57.072512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08833c1-dccb-4588-bc74-b5c6e58d4c29 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:57.074099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 593/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:56 2022] GET /identity/v3/users/48aae7da46d44547b65e5c01dd7fd740 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:57.079635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94b261-eead-410c-bf6f-18ca074f1b87 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:57.173723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94b261-eead-410c-bf6f-18ca074f1b87 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:57.205778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e6101-e3e6-4449-bd9f-e847a7f55004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:57.206667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e6101-e3e6-4449-bd9f-e847a7f55004 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:57.206953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e6101-e3e6-4449-bd9f-e847a7f55004 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:57.207229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e6101-e3e6-4449-bd9f-e847a7f55004 None admin] PATH_INFO: `/v3/projects/6696d1c87c26481d8def5b6090b6bdfb` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:57.208384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e6101-e3e6-4449-bd9f-e847a7f55004 None admin] RBAC: Authorizing `identity:get_project(project_id=6696d1c87c26481d8def5b6090b6bdfb)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:57.209777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e6101-e3e6-4449-bd9f-e847a7f55004 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:57.211442 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 598/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:52:57 2022] GET /identity/v3/projects/6696d1c87c26481d8def5b6090b6bdfb => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:57.217000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08833c1-dccb-4588-bc74-b5c6e58d4c29 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:57.312147 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08833c1-dccb-4588-bc74-b5c6e58d4c29 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:57.344056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e891598-302f-4a80-8076-b9a2461865c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:57.344921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e891598-302f-4a80-8076-b9a2461865c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:57.345203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e891598-302f-4a80-8076-b9a2461865c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:57.345478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e891598-302f-4a80-8076-b9a2461865c3 None admin] PATH_INFO: `/v3/projects/6696d1c87c26481d8def5b6090b6bdfb/users/48aae7da46d44547b65e5c01dd7fd740/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:57.347467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e891598-302f-4a80-8076-b9a2461865c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6696d1c87c26481d8def5b6090b6bdfb, user_id=48aae7da46d44547b65e5c01dd7fd740, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:57.348896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e891598-302f-4a80-8076-b9a2461865c3 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:57.356707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:57.362605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 594/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:52:57 2022] PUT /identity/v3/projects/6696d1c87c26481d8def5b6090b6bdfb/users/48aae7da46d44547b65e5c01dd7fd740/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:52:59.498658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8b447-50cc-4b02-8114-761323d8c6ad None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:59.499210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8b447-50cc-4b02-8114-761323d8c6ad None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:59.499210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8b447-50cc-4b02-8114-761323d8c6ad None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:59.500575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 599/1193] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:52:59.505858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d3efb-93d8-4317-a2d6-f5aef4078159 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:59.506258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d3efb-93d8-4317-a2d6-f5aef4078159 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:59.506421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d3efb-93d8-4317-a2d6-f5aef4078159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:59.523583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-851d3efb-93d8-4317-a2d6-f5aef4078159 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:52:59.524762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-851d3efb-93d8-4317-a2d6-f5aef4078159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:52:59.588918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 595/1194] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:52:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:52:59.612468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8b447-50cc-4b02-8114-761323d8c6ad None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:59.624063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:59.713339 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8b447-50cc-4b02-8114-761323d8c6ad None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:59.745013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae6671-917b-4afb-955d-423d8b93900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:59.745793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae6671-917b-4afb-955d-423d8b93900b None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:59.746082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae6671-917b-4afb-955d-423d8b93900b None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:59.746362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae6671-917b-4afb-955d-423d8b93900b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:59.757207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae6671-917b-4afb-955d-423d8b93900b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-eaae6671-917b-4afb-955d-423d8b93900b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:52:59.765086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:52:59.766204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:52:59.767313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 600/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:52:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:52:59.772869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d3efb-93d8-4317-a2d6-f5aef4078159 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:59.866923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d3efb-93d8-4317-a2d6-f5aef4078159 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:52:59.892264 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:52:59.900610 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586c1fa-4bc0-45cd-a7bc-ebb3e9ca7b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:52:59.901266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586c1fa-4bc0-45cd-a7bc-ebb3e9ca7b2c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:52:59.901551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586c1fa-4bc0-45cd-a7bc-ebb3e9ca7b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:52:59.901829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586c1fa-4bc0-45cd-a7bc-ebb3e9ca7b2c None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:52:59.902598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586c1fa-4bc0-45cd-a7bc-ebb3e9ca7b2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:52:59.902909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586c1fa-4bc0-45cd-a7bc-ebb3e9ca7b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:52:59.904286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586c1fa-4bc0-45cd-a7bc-ebb3e9ca7b2c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:52:59.912063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 596/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:52:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:52:59.917814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae6671-917b-4afb-955d-423d8b93900b None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:52:59.983383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:53:00.015983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae6671-917b-4afb-955d-423d8b93900b None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:53:00.048152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d987a-1971-454c-95fc-c3d88e3ca548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:53:00.049016 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d987a-1971-454c-95fc-c3d88e3ca548 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:53:00.049297 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d987a-1971-454c-95fc-c3d88e3ca548 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:53:00.049572 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d987a-1971-454c-95fc-c3d88e3ca548 None admin] PATH_INFO: `/v3/users/48aae7da46d44547b65e5c01dd7fd740` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:53:00.051282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d987a-1971-454c-95fc-c3d88e3ca548 None admin] RBAC: Authorizing `identity:get_user(user_id=48aae7da46d44547b65e5c01dd7fd740)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:53:00.052703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d987a-1971-454c-95fc-c3d88e3ca548 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:53:00.054298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 601/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Thu May 26 20:52:59 2022] GET /identity/v3/users/48aae7da46d44547b65e5c01dd7fd740 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:53:00.060069 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586c1fa-4bc0-45cd-a7bc-ebb3e9ca7b2c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:53:00.155131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586c1fa-4bc0-45cd-a7bc-ebb3e9ca7b2c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:53:00.186847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18377bf5-9ee0-4de0-a122-54bf5ef41688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:53:00.187472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377bf5-9ee0-4de0-a122-54bf5ef41688 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:53:00.187752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377bf5-9ee0-4de0-a122-54bf5ef41688 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:53:00.188032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377bf5-9ee0-4de0-a122-54bf5ef41688 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:53:00.241655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:53:00.252871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18377bf5-9ee0-4de0-a122-54bf5ef41688 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:53:00.254930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18377bf5-9ee0-4de0-a122-54bf5ef41688 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:53:00.347970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 597/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:53:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:53:00.353938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d987a-1971-454c-95fc-c3d88e3ca548 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:53:00.449387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d987a-1971-454c-95fc-c3d88e3ca548 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:53:00.482181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825edd0d-4e79-4891-a88b-330444ab04a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:53:00.483060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825edd0d-4e79-4891-a88b-330444ab04a0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:53:00.483347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825edd0d-4e79-4891-a88b-330444ab04a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:53:00.483626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825edd0d-4e79-4891-a88b-330444ab04a0 None admin] PATH_INFO: `/v3/projects/6696d1c87c26481d8def5b6090b6bdfb` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:53:00.484794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825edd0d-4e79-4891-a88b-330444ab04a0 None admin] RBAC: Authorizing `identity:get_project(project_id=6696d1c87c26481d8def5b6090b6bdfb)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:53:00.486272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825edd0d-4e79-4891-a88b-330444ab04a0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:53:00.488609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 602/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Thu May 26 20:53:00 2022] GET /identity/v3/projects/6696d1c87c26481d8def5b6090b6bdfb => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:53:00.494587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18377bf5-9ee0-4de0-a122-54bf5ef41688 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:53:00.579304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:53:00.589020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18377bf5-9ee0-4de0-a122-54bf5ef41688 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:53:00.622246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc7be2-8937-4164-af55-ecc1c19f7183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:53:00.622869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc7be2-8937-4164-af55-ecc1c19f7183 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:53:00.623151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc7be2-8937-4164-af55-ecc1c19f7183 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:53:00.623425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc7be2-8937-4164-af55-ecc1c19f7183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:53:00.624237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc7be2-8937-4164-af55-ecc1c19f7183 None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=6696d1c87c26481d8def5b6090b6bdfb, user.id=48aae7da46d44547b65e5c01dd7fd740) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:53:00.624546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc7be2-8937-4164-af55-ecc1c19f7183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:53:00.625973 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc7be2-8937-4164-af55-ecc1c19f7183 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:53:00.631602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 598/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:53:00 2022] GET /identity/v3/role_assignments?user.id=48aae7da46d44547b65e5c01dd7fd740&scope.project.id=6696d1c87c26481d8def5b6090b6bdfb&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:53:10.478436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:53:10.629081 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=23 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:00.423031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c495d-9212-43f1-8d91-272e1d976433 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:00.423031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c495d-9212-43f1-8d91-272e1d976433 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:00.423031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c495d-9212-43f1-8d91-272e1d976433 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:00.423031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 603/1201] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:54:00.425579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d0d85-8768-48fc-954a-2b72f83d0c0a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:00.426024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d0d85-8768-48fc-954a-2b72f83d0c0a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:00.426206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d0d85-8768-48fc-954a-2b72f83d0c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:00.447028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0f9d0d85-8768-48fc-954a-2b72f83d0c0a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:00.448483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0f9d0d85-8768-48fc-954a-2b72f83d0c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:00.544330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 599/1202] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:00.577422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c495d-9212-43f1-8d91-272e1d976433 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:00.718897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c495d-9212-43f1-8d91-272e1d976433 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:00.761124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345dd86-1798-4689-a667-2910989d0405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:00.761920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345dd86-1798-4689-a667-2910989d0405 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:00.762256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345dd86-1798-4689-a667-2910989d0405 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:00.762502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345dd86-1798-4689-a667-2910989d0405 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:00.781162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345dd86-1798-4689-a667-2910989d0405 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:00.783115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345dd86-1798-4689-a667-2910989d0405 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2345dd86-1798-4689-a667-2910989d0405 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:00.806662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:00.807809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:00.808902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:00.808902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:00.808902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:00.808902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:00.808902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 604/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:00.815073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d0d85-8768-48fc-954a-2b72f83d0c0a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:00.961085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d0d85-8768-48fc-954a-2b72f83d0c0a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:00.993522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:01.000068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36bd48-2591-434d-a1bc-bf5785cc31f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:01.000904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36bd48-2591-434d-a1bc-bf5785cc31f7 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:01.001227 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36bd48-2591-434d-a1bc-bf5785cc31f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:01.001934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36bd48-2591-434d-a1bc-bf5785cc31f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:01.003000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36bd48-2591-434d-a1bc-bf5785cc31f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:01.003492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36bd48-2591-434d-a1bc-bf5785cc31f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:01.004909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36bd48-2591-434d-a1bc-bf5785cc31f7 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:01.029563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345dd86-1798-4689-a667-2910989d0405 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:01.030300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 600/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:01.094102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:01.147773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345dd86-1798-4689-a667-2910989d0405 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:01.191977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da07417-dc4c-429a-ae1e-32eb30dbd431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:01.192205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da07417-dc4c-429a-ae1e-32eb30dbd431 None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:01.192379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da07417-dc4c-429a-ae1e-32eb30dbd431 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:01.192607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da07417-dc4c-429a-ae1e-32eb30dbd431 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:01.193320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da07417-dc4c-429a-ae1e-32eb30dbd431 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:01.194787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da07417-dc4c-429a-ae1e-32eb30dbd431 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:01.220493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 605/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Thu May 26 20:54:01 2022] POST /identity/v3/users => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 20:54:03.532038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61849201-e269-4c1f-b555-6313d84fa945 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:03.532582 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61849201-e269-4c1f-b555-6313d84fa945 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:03.532582 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61849201-e269-4c1f-b555-6313d84fa945 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:03.534219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 601/1206] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:54:03.539626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1085-10b0-4aaf-9645-7249bcda83f0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:03.540129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1085-10b0-4aaf-9645-7249bcda83f0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:03.540238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f1085-10b0-4aaf-9645-7249bcda83f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:03.559875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2f5f1085-10b0-4aaf-9645-7249bcda83f0 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:03.561061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2f5f1085-10b0-4aaf-9645-7249bcda83f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:03.647861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 606/1207] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:03.672773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61849201-e269-4c1f-b555-6313d84fa945 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:03.816939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61849201-e269-4c1f-b555-6313d84fa945 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:03.862242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9c7d3-9f57-43c2-9646-cf611908beaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:03.863286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9c7d3-9f57-43c2-9646-cf611908beaf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:03.863634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9c7d3-9f57-43c2-9646-cf611908beaf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:03.863948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c9c7d3-9f57-43c2-9646-cf611908beaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:03.875442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c9c7d3-9f57-43c2-9646-cf611908beaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-18c9c7d3-9f57-43c2-9646-cf611908beaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:54:03.884094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:03.885104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:03.886169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 602/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:03.892109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f1085-10b0-4aaf-9645-7249bcda83f0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:03.938925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:04.011569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f1085-10b0-4aaf-9645-7249bcda83f0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:04.052280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d714a-8a15-4525-9541-9e462f7db39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:04.053284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d714a-8a15-4525-9541-9e462f7db39f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:04.053623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d714a-8a15-4525-9541-9e462f7db39f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:04.053916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d714a-8a15-4525-9541-9e462f7db39f None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:04.054774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d714a-8a15-4525-9541-9e462f7db39f None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:04.055093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d714a-8a15-4525-9541-9e462f7db39f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:04.059252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d714a-8a15-4525-9541-9e462f7db39f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:04.073088 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 607/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:54:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:04.075000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9c7d3-9f57-43c2-9646-cf611908beaf None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:04.133775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:04.197789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c9c7d3-9f57-43c2-9646-cf611908beaf None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:04.233808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cee2d0-b8f7-4e14-8ac7-cdbc0fb3129f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:04.234613 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cee2d0-b8f7-4e14-8ac7-cdbc0fb3129f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:04.234894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cee2d0-b8f7-4e14-8ac7-cdbc0fb3129f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:04.235782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cee2d0-b8f7-4e14-8ac7-cdbc0fb3129f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:04.249856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cee2d0-b8f7-4e14-8ac7-cdbc0fb3129f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:04.251544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cee2d0-b8f7-4e14-8ac7-cdbc0fb3129f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-a6cee2d0-b8f7-4e14-8ac7-cdbc0fb3129f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:04.271854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:04.273133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:04.274369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:04.274369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:04.274369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:04.274369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:04.278094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 603/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:04.279666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d714a-8a15-4525-9541-9e462f7db39f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:04.395578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:04.398728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d714a-8a15-4525-9541-9e462f7db39f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:04.437372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c42d4a-aa13-49d2-b920-4c1f45480d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:04.438059 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c42d4a-aa13-49d2-b920-4c1f45480d66 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:04.438344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c42d4a-aa13-49d2-b920-4c1f45480d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:04.438657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c42d4a-aa13-49d2-b920-4c1f45480d66 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:04.439377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c42d4a-aa13-49d2-b920-4c1f45480d66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:04.439685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c42d4a-aa13-49d2-b920-4c1f45480d66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:04.441055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c42d4a-aa13-49d2-b920-4c1f45480d66 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:04.453093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 608/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 20:54:04.459553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cee2d0-b8f7-4e14-8ac7-cdbc0fb3129f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:04.587850 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cee2d0-b8f7-4e14-8ac7-cdbc0fb3129f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:04.590853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:04.630743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51e7fd-629e-48e1-bb81-2e6eb847d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:04.632121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51e7fd-629e-48e1-bb81-2e6eb847d715 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:04.632359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51e7fd-629e-48e1-bb81-2e6eb847d715 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:04.632658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd51e7fd-629e-48e1-bb81-2e6eb847d715 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:04.641875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51e7fd-629e-48e1-bb81-2e6eb847d715 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:04.643376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd51e7fd-629e-48e1-bb81-2e6eb847d715 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-dd51e7fd-629e-48e1-bb81-2e6eb847d715 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:54:04.660271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:04.661608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:04.662808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 604/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:04.665671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c42d4a-aa13-49d2-b920-4c1f45480d66 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:04.780167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c42d4a-aa13-49d2-b920-4c1f45480d66 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:04.822813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e560e-91cb-4036-a95d-ce09c2949127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:04.823648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e560e-91cb-4036-a95d-ce09c2949127 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:04.823945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e560e-91cb-4036-a95d-ce09c2949127 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:04.824238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e560e-91cb-4036-a95d-ce09c2949127 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:04.825309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e560e-91cb-4036-a95d-ce09c2949127 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:04.825675 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e560e-91cb-4036-a95d-ce09c2949127 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:04.827347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e560e-91cb-4036-a95d-ce09c2949127 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:04.848197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 609/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 26 20:54:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:04.854166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51e7fd-629e-48e1-bb81-2e6eb847d715 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:04.969978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd51e7fd-629e-48e1-bb81-2e6eb847d715 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:05.009515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b8663-3af9-4a59-8d43-6c71daf28aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:05.010239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8663-3af9-4a59-8d43-6c71daf28aee None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:05.010618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8663-3af9-4a59-8d43-6c71daf28aee None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:05.010906 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b8663-3af9-4a59-8d43-6c71daf28aee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:05.046019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:05.096065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b8663-3af9-4a59-8d43-6c71daf28aee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:05.098061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7b8663-3af9-4a59-8d43-6c71daf28aee None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:05.230436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 605/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:54:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:54:05.236447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e560e-91cb-4036-a95d-ce09c2949127 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:05.257467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:05.357014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e560e-91cb-4036-a95d-ce09c2949127 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:05.394972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee68a14-a041-45ee-b9cb-3f9ddff19e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:05.395786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee68a14-a041-45ee-b9cb-3f9ddff19e91 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:05.396077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee68a14-a041-45ee-b9cb-3f9ddff19e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:05.396359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee68a14-a041-45ee-b9cb-3f9ddff19e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:05.407856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee68a14-a041-45ee-b9cb-3f9ddff19e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:05.409305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee68a14-a041-45ee-b9cb-3f9ddff19e91 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-eee68a14-a041-45ee-b9cb-3f9ddff19e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:05.431944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:05.433208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:05.438817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:05.438817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:05.438817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:05.438817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:05.438817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 610/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:05.439255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b8663-3af9-4a59-8d43-6c71daf28aee None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:05.515124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:05.564694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b8663-3af9-4a59-8d43-6c71daf28aee None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:05.617431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f228f62-c5f7-464b-9229-57abb4d23394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:05.618315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f228f62-c5f7-464b-9229-57abb4d23394 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:05.618637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f228f62-c5f7-464b-9229-57abb4d23394 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:05.618980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f228f62-c5f7-464b-9229-57abb4d23394 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:05.619982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f228f62-c5f7-464b-9229-57abb4d23394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:05.620480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f228f62-c5f7-464b-9229-57abb4d23394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:05.622014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f228f62-c5f7-464b-9229-57abb4d23394 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:05.636320 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 606/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:05.642245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee68a14-a041-45ee-b9cb-3f9ddff19e91 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:05.722373 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:05.762835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee68a14-a041-45ee-b9cb-3f9ddff19e91 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:05.808700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ad62b-b466-4056-bbe0-d21234470eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:05.809911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ad62b-b466-4056-bbe0-d21234470eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:05.810430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ad62b-b466-4056-bbe0-d21234470eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:05.810983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ad62b-b466-4056-bbe0-d21234470eb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:05.821909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ad62b-b466-4056-bbe0-d21234470eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:05.824270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ad62b-b466-4056-bbe0-d21234470eb0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a68ad62b-b466-4056-bbe0-d21234470eb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:54:05.843911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:05.845241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:05.846384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:54:05.846384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:54:05.846384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:54:05.846384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:54:05.846384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:54:05.846384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:54:05.846384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:05.846384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:05.848660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 611/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:54:05.854876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f228f62-c5f7-464b-9229-57abb4d23394 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:05.992567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f228f62-c5f7-464b-9229-57abb4d23394 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:06.019986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:06.033799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b64ed1-7950-4f11-820e-05cd3aa06899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:06.035064 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b64ed1-7950-4f11-820e-05cd3aa06899 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:06.035677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b64ed1-7950-4f11-820e-05cd3aa06899 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:06.036082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b64ed1-7950-4f11-820e-05cd3aa06899 None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:06.037129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b64ed1-7950-4f11-820e-05cd3aa06899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:06.037477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b64ed1-7950-4f11-820e-05cd3aa06899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:06.039426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b64ed1-7950-4f11-820e-05cd3aa06899 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:06.059662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 607/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:54:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:06.061815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ad62b-b466-4056-bbe0-d21234470eb0 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:06.198540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ad62b-b466-4056-bbe0-d21234470eb0 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:06.236365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:06.245225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b5ecb-c813-4465-90f5-86833f353c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:06.246343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b5ecb-c813-4465-90f5-86833f353c02 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:06.246829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b5ecb-c813-4465-90f5-86833f353c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:06.247452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b5ecb-c813-4465-90f5-86833f353c02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:06.248696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b5ecb-c813-4465-90f5-86833f353c02 None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=4edb53dc871047ce89ebb3b90143a6de) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:06.249202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b5ecb-c813-4465-90f5-86833f353c02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:06.251168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b5ecb-c813-4465-90f5-86833f353c02 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:06.265223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 612/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:54:06 2022] GET /identity/v3/role_assignments?user.id=4edb53dc871047ce89ebb3b90143a6de&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:08.580397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961df34-3a8c-453a-8a65-545d1a32c81e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:08.580967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961df34-3a8c-453a-8a65-545d1a32c81e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:08.580967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961df34-3a8c-453a-8a65-545d1a32c81e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:08.582511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 608/1220] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:54:08.588235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aea671-c00e-47b0-addc-2a72756534cf None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:08.588783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aea671-c00e-47b0-addc-2a72756534cf None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:08.588783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aea671-c00e-47b0-addc-2a72756534cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:08.610227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d4aea671-c00e-47b0-addc-2a72756534cf None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:08.611705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d4aea671-c00e-47b0-addc-2a72756534cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:08.699426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 613/1221] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:08.722217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961df34-3a8c-453a-8a65-545d1a32c81e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:08.842873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961df34-3a8c-453a-8a65-545d1a32c81e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:08.878721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8328e-0003-4b23-92ce-b04b2d075710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:08.879513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8328e-0003-4b23-92ce-b04b2d075710 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:08.879798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8328e-0003-4b23-92ce-b04b2d075710 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:08.880074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8328e-0003-4b23-92ce-b04b2d075710 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:08.890219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8328e-0003-4b23-92ce-b04b2d075710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-dde8328e-0003-4b23-92ce-b04b2d075710 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:54:08.898615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:08.899964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:08.901314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 609/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:54:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:08.907516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aea671-c00e-47b0-addc-2a72756534cf None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:09.028177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aea671-c00e-47b0-addc-2a72756534cf None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:09.053298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:09.065267 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979316c-004a-4b32-8900-80707afec3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:09.065937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979316c-004a-4b32-8900-80707afec3ea None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:09.066228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979316c-004a-4b32-8900-80707afec3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:09.066537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979316c-004a-4b32-8900-80707afec3ea None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:09.067280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979316c-004a-4b32-8900-80707afec3ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:09.067589 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979316c-004a-4b32-8900-80707afec3ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:09.068972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979316c-004a-4b32-8900-80707afec3ea None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:09.078022 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 614/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:54:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:09.084162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8328e-0003-4b23-92ce-b04b2d075710 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:09.128119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:09.185844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8328e-0003-4b23-92ce-b04b2d075710 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:09.221924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69741307-6878-4381-9c66-354971aa597a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:09.222829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69741307-6878-4381-9c66-354971aa597a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:09.223108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69741307-6878-4381-9c66-354971aa597a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:09.223384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69741307-6878-4381-9c66-354971aa597a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:09.233817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69741307-6878-4381-9c66-354971aa597a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:09.235330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69741307-6878-4381-9c66-354971aa597a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-69741307-6878-4381-9c66-354971aa597a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:09.252257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:09.253576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:09.254709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:09.254709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:09.254709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:09.254709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:09.254709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 610/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:09.260885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979316c-004a-4b32-8900-80707afec3ea None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:09.374917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979316c-004a-4b32-8900-80707afec3ea None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:09.418180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d93bcd-5246-4cca-b140-b3dc01791e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:09.419013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93bcd-5246-4cca-b140-b3dc01791e96 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:09.419349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93bcd-5246-4cca-b140-b3dc01791e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:09.419631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93bcd-5246-4cca-b140-b3dc01791e96 None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:09.420586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d93bcd-5246-4cca-b140-b3dc01791e96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:09.420937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d93bcd-5246-4cca-b140-b3dc01791e96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:09.422865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d93bcd-5246-4cca-b140-b3dc01791e96 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:09.436704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 615/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:09.443096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69741307-6878-4381-9c66-354971aa597a None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:09.555995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:09.566948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69741307-6878-4381-9c66-354971aa597a None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:09.619591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e18b2-45dd-4bfe-b541-fa1606c232ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:09.620568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e18b2-45dd-4bfe-b541-fa1606c232ac None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:09.620985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e18b2-45dd-4bfe-b541-fa1606c232ac None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:09.621400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e18b2-45dd-4bfe-b541-fa1606c232ac None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:09.630675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e18b2-45dd-4bfe-b541-fa1606c232ac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:09.632470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e18b2-45dd-4bfe-b541-fa1606c232ac None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-d88e18b2-45dd-4bfe-b541-fa1606c232ac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:54:09.648775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:09.649923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:09.651076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:09.651076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:09.651076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:54:09.651076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:54:09.651076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:54:09.651076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:54:09.651076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:54:09.651076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:54:09.651076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:09.651076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:09.655028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 611/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:09.656790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d93bcd-5246-4cca-b140-b3dc01791e96 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:09.715655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:09.772378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d93bcd-5246-4cca-b140-b3dc01791e96 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:09.809927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7925e5-6d80-4e3a-b046-4bcfc8fd7583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:09.810636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7925e5-6d80-4e3a-b046-4bcfc8fd7583 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:09.810923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7925e5-6d80-4e3a-b046-4bcfc8fd7583 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:09.811203 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7925e5-6d80-4e3a-b046-4bcfc8fd7583 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:09.812046 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7925e5-6d80-4e3a-b046-4bcfc8fd7583 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:09.812360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7925e5-6d80-4e3a-b046-4bcfc8fd7583 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:09.814480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7925e5-6d80-4e3a-b046-4bcfc8fd7583 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:09.832928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 616/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 26 20:54:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:09.838575 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e18b2-45dd-4bfe-b541-fa1606c232ac None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:09.948055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e18b2-45dd-4bfe-b541-fa1606c232ac None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:09.983426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef60c52-1ebd-42aa-ac1e-4dd67c62f33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:09.984144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef60c52-1ebd-42aa-ac1e-4dd67c62f33d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:09.984493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef60c52-1ebd-42aa-ac1e-4dd67c62f33d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:09.985222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef60c52-1ebd-42aa-ac1e-4dd67c62f33d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:09.995673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef60c52-1ebd-42aa-ac1e-4dd67c62f33d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:09.997195 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef60c52-1ebd-42aa-ac1e-4dd67c62f33d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-9ef60c52-1ebd-42aa-ac1e-4dd67c62f33d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:10.018577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:10.019578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:10.020503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:10.020503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:10.020503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:10.020503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:10.020503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 612/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 20:54:10.025708 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7925e5-6d80-4e3a-b046-4bcfc8fd7583 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:10.130820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7925e5-6d80-4e3a-b046-4bcfc8fd7583 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:10.133837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:10.169478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f769d-d194-4315-822a-07c7ba7475fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:10.170187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f769d-d194-4315-822a-07c7ba7475fa None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:10.170624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f769d-d194-4315-822a-07c7ba7475fa None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:10.171108 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f769d-d194-4315-822a-07c7ba7475fa None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:10.171897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f769d-d194-4315-822a-07c7ba7475fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:10.172864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f769d-d194-4315-822a-07c7ba7475fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:10.174547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505f769d-d194-4315-822a-07c7ba7475fa None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:10.188750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 617/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:10.195276 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef60c52-1ebd-42aa-ac1e-4dd67c62f33d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:10.207948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:10.313483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef60c52-1ebd-42aa-ac1e-4dd67c62f33d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:10.355710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d698a8-84cc-4da8-ac75-48505ce7eb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:10.356505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d698a8-84cc-4da8-ac75-48505ce7eb15 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:10.356784 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d698a8-84cc-4da8-ac75-48505ce7eb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:10.357058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d698a8-84cc-4da8-ac75-48505ce7eb15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:10.364457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d698a8-84cc-4da8-ac75-48505ce7eb15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:10.365910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d698a8-84cc-4da8-ac75-48505ce7eb15 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-e4d698a8-84cc-4da8-ac75-48505ce7eb15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:54:10.381459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:10.382432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:10.383658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:54:10.383658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:54:10.383658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:54:10.383658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:54:10.383658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:54:10.383658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:54:10.383658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:10.383658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:10.383658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 613/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:54:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:10.388922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f769d-d194-4315-822a-07c7ba7475fa None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:10.491466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f769d-d194-4315-822a-07c7ba7475fa None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:10.526775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45cf2b6-6c13-4a44-8f6b-47329352f70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:10.527482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cf2b6-6c13-4a44-8f6b-47329352f70c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:10.527799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cf2b6-6c13-4a44-8f6b-47329352f70c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:10.528083 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cf2b6-6c13-4a44-8f6b-47329352f70c None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:10.528921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cf2b6-6c13-4a44-8f6b-47329352f70c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:10.529235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cf2b6-6c13-4a44-8f6b-47329352f70c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:10.530806 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45cf2b6-6c13-4a44-8f6b-47329352f70c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:10.549424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 618/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:54:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:10.550635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d698a8-84cc-4da8-ac75-48505ce7eb15 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:10.622587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:10.663616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d698a8-84cc-4da8-ac75-48505ce7eb15 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:10.701040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a20fdd-7e4a-4391-b196-15eb7e30d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:10.702146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a20fdd-7e4a-4391-b196-15eb7e30d78f None admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:10.702489 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a20fdd-7e4a-4391-b196-15eb7e30d78f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:10.702872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a20fdd-7e4a-4391-b196-15eb7e30d78f None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/4edb53dc871047ce89ebb3b90143a6de/roles/4e24596391d3466eacb50a9e923e07ab` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:10.719340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a20fdd-7e4a-4391-b196-15eb7e30d78f None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=4edb53dc871047ce89ebb3b90143a6de, role_id=4e24596391d3466eacb50a9e923e07ab)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:10.720814 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a20fdd-7e4a-4391-b196-15eb7e30d78f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:10.736194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 614/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:54:10 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/4edb53dc871047ce89ebb3b90143a6de/roles/4e24596391d3466eacb50a9e923e07ab => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 20:54:13.170107 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6044c-1129-4bb5-b133-c60400950a78 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:13.170684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6044c-1129-4bb5-b133-c60400950a78 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:13.170684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6044c-1129-4bb5-b133-c60400950a78 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:13.174477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 619/1233] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:54:13.178195 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54b687-e613-4ebd-9a5d-7bd1c7f2af0b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:13.178689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54b687-e613-4ebd-9a5d-7bd1c7f2af0b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:13.178894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54b687-e613-4ebd-9a5d-7bd1c7f2af0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:13.197087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9e54b687-e613-4ebd-9a5d-7bd1c7f2af0b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:13.198433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9e54b687-e613-4ebd-9a5d-7bd1c7f2af0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:13.283174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 615/1234] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:13.313945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6044c-1129-4bb5-b133-c60400950a78 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:13.344407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:13.433586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6044c-1129-4bb5-b133-c60400950a78 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:13.471345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c481862-6f60-4e0c-bf4e-a68a762d93a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:13.472240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c481862-6f60-4e0c-bf4e-a68a762d93a9 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:13.472592 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c481862-6f60-4e0c-bf4e-a68a762d93a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:13.472937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c481862-6f60-4e0c-bf4e-a68a762d93a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:13.483213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c481862-6f60-4e0c-bf4e-a68a762d93a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1c481862-6f60-4e0c-bf4e-a68a762d93a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:54:13.491429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:13.492755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:13.493960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 620/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Thu May 26 20:54:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:13.498580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54b687-e613-4ebd-9a5d-7bd1c7f2af0b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:13.547998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:13.610647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54b687-e613-4ebd-9a5d-7bd1c7f2af0b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:13.649307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514134fc-83a6-4cb0-8d94-2b67c19c82b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:13.650019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514134fc-83a6-4cb0-8d94-2b67c19c82b1 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:13.650342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514134fc-83a6-4cb0-8d94-2b67c19c82b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:13.650679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514134fc-83a6-4cb0-8d94-2b67c19c82b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:13.651494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514134fc-83a6-4cb0-8d94-2b67c19c82b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:13.651849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514134fc-83a6-4cb0-8d94-2b67c19c82b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:13.653432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514134fc-83a6-4cb0-8d94-2b67c19c82b1 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:13.663498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 616/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Thu May 26 20:54:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:13.670075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c481862-6f60-4e0c-bf4e-a68a762d93a9 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:13.767350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:13.785457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c481862-6f60-4e0c-bf4e-a68a762d93a9 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:13.826623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf1461-e237-48bb-805b-99f761884e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:13.827423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf1461-e237-48bb-805b-99f761884e07 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:13.827709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf1461-e237-48bb-805b-99f761884e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:13.827983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf1461-e237-48bb-805b-99f761884e07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:13.838920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf1461-e237-48bb-805b-99f761884e07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:13.840368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf1461-e237-48bb-805b-99f761884e07 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-b6cf1461-e237-48bb-805b-99f761884e07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:13.858939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:13.860127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:13.861247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:13.861247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:13.861247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:13.861247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:13.861247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 621/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:13.865929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514134fc-83a6-4cb0-8d94-2b67c19c82b1 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:13.978029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514134fc-83a6-4cb0-8d94-2b67c19c82b1 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:13.988687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:14.017721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0404e05-9607-41a0-a344-1a290b75ac18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:14.018392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0404e05-9607-41a0-a344-1a290b75ac18 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:14.018714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0404e05-9607-41a0-a344-1a290b75ac18 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:14.018994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0404e05-9607-41a0-a344-1a290b75ac18 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:14.019843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0404e05-9607-41a0-a344-1a290b75ac18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:14.020179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0404e05-9607-41a0-a344-1a290b75ac18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:14.021841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0404e05-9607-41a0-a344-1a290b75ac18 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:14.035528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 617/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:14.041474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf1461-e237-48bb-805b-99f761884e07 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:14.150648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf1461-e237-48bb-805b-99f761884e07 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:14.190335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e9b7c-01c8-4f99-9c31-3000ecabd49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:14.191389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e9b7c-01c8-4f99-9c31-3000ecabd49c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:14.191705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e9b7c-01c8-4f99-9c31-3000ecabd49c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:14.191999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e9b7c-01c8-4f99-9c31-3000ecabd49c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:14.195212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:14.201750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e9b7c-01c8-4f99-9c31-3000ecabd49c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:14.203947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e9b7c-01c8-4f99-9c31-3000ecabd49c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-679e9b7c-01c8-4f99-9c31-3000ecabd49c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:54:14.219775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:14.220966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:14.222105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 622/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:14.227165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0404e05-9607-41a0-a344-1a290b75ac18 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:14.337923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0404e05-9607-41a0-a344-1a290b75ac18 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:14.371932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a80da-0f19-455b-b88c-3974e42b5b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:14.372598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a80da-0f19-455b-b88c-3974e42b5b3c None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:14.372910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a80da-0f19-455b-b88c-3974e42b5b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:14.373187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a80da-0f19-455b-b88c-3974e42b5b3c None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:14.374005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a80da-0f19-455b-b88c-3974e42b5b3c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:14.374316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a80da-0f19-455b-b88c-3974e42b5b3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:14.376368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a80da-0f19-455b-b88c-3974e42b5b3c None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:14.395380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 618/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 26 20:54:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:14.402570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e9b7c-01c8-4f99-9c31-3000ecabd49c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:14.508933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e9b7c-01c8-4f99-9c31-3000ecabd49c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:14.542849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11935891-f058-4a61-a116-a9e559628ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:14.543498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11935891-f058-4a61-a116-a9e559628ffd None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:14.543809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11935891-f058-4a61-a116-a9e559628ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:14.544052 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11935891-f058-4a61-a116-a9e559628ffd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:14.618823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:14.628189 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11935891-f058-4a61-a116-a9e559628ffd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:14.629785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11935891-f058-4a61-a116-a9e559628ffd None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:14.742492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 623/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:54:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:54:14.746269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a80da-0f19-455b-b88c-3974e42b5b3c None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:14.777130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:14.856341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a80da-0f19-455b-b88c-3974e42b5b3c None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:14.890691 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f68b9f-7c2f-4a5b-a45a-dc6828b5d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:14.891397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f68b9f-7c2f-4a5b-a45a-dc6828b5d13d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:14.891677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f68b9f-7c2f-4a5b-a45a-dc6828b5d13d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:14.891951 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f68b9f-7c2f-4a5b-a45a-dc6828b5d13d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:14.902817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f68b9f-7c2f-4a5b-a45a-dc6828b5d13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:14.904279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f68b9f-7c2f-4a5b-a45a-dc6828b5d13d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-16f68b9f-7c2f-4a5b-a45a-dc6828b5d13d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:14.926380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:14.927399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:14.928326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:14.928326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:14.928326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:14.928326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:14.928326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 619/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:14.934002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11935891-f058-4a61-a116-a9e559628ffd None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:15.037323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:15.043963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11935891-f058-4a61-a116-a9e559628ffd None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:15.081722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619ded4-b60f-49fd-a7ab-771691d7acae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:15.082583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619ded4-b60f-49fd-a7ab-771691d7acae None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:15.082918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619ded4-b60f-49fd-a7ab-771691d7acae None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:15.083192 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619ded4-b60f-49fd-a7ab-771691d7acae None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:15.084027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619ded4-b60f-49fd-a7ab-771691d7acae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:15.084353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619ded4-b60f-49fd-a7ab-771691d7acae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:15.085879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619ded4-b60f-49fd-a7ab-771691d7acae None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:15.099418 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 624/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:15.105583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f68b9f-7c2f-4a5b-a45a-dc6828b5d13d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:15.194840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:15.220302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f68b9f-7c2f-4a5b-a45a-dc6828b5d13d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:15.254710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8e90c-c3bd-4c7f-b891-7f5dd47c7272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:15.255572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8e90c-c3bd-4c7f-b891-7f5dd47c7272 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:15.255855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8e90c-c3bd-4c7f-b891-7f5dd47c7272 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:15.256383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c8e90c-c3bd-4c7f-b891-7f5dd47c7272 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:15.263910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8e90c-c3bd-4c7f-b891-7f5dd47c7272 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:15.265407 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c8e90c-c3bd-4c7f-b891-7f5dd47c7272 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-44c8e90c-c3bd-4c7f-b891-7f5dd47c7272 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:54:15.279475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:15.280690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:15.282169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:54:15.282169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:54:15.282169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:54:15.282169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:54:15.282169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:54:15.282169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:54:15.282169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:15.282169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:15.282169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 620/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:54:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:15.287567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619ded4-b60f-49fd-a7ab-771691d7acae None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:15.397090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619ded4-b60f-49fd-a7ab-771691d7acae None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:15.434817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768eed31-029f-44ae-a45c-44e9e897604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:15.435480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768eed31-029f-44ae-a45c-44e9e897604d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:15.435768 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768eed31-029f-44ae-a45c-44e9e897604d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:15.436041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768eed31-029f-44ae-a45c-44e9e897604d None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:15.436852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768eed31-029f-44ae-a45c-44e9e897604d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:15.437160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768eed31-029f-44ae-a45c-44e9e897604d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:15.439183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768eed31-029f-44ae-a45c-44e9e897604d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:15.452645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 625/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:54:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:15.459215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8e90c-c3bd-4c7f-b891-7f5dd47c7272 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:15.581362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c8e90c-c3bd-4c7f-b891-7f5dd47c7272 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:15.631838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5516b4e-81a5-4b6e-9e03-7cc9b90e962f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:15.632581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5516b4e-81a5-4b6e-9e03-7cc9b90e962f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:15.632886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5516b4e-81a5-4b6e-9e03-7cc9b90e962f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:15.633189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5516b4e-81a5-4b6e-9e03-7cc9b90e962f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:15.634156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5516b4e-81a5-4b6e-9e03-7cc9b90e962f None admin] RBAC: Adding query filter params (role.id=4e24596391d3466eacb50a9e923e07ab, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=4edb53dc871047ce89ebb3b90143a6de) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:15.634566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5516b4e-81a5-4b6e-9e03-7cc9b90e962f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:15.636230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5516b4e-81a5-4b6e-9e03-7cc9b90e962f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:15.643588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 621/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:54:15 2022] GET /identity/v3/role_assignments?user.id=4edb53dc871047ce89ebb3b90143a6de&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=4e24596391d3466eacb50a9e923e07ab => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:18.027897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c67c3-243b-45f2-9f5f-31c9459cdfd6 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:18.028501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c67c3-243b-45f2-9f5f-31c9459cdfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:18.028501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c67c3-243b-45f2-9f5f-31c9459cdfd6 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:18.029925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 626/1247] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:54:18.036158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f34376-2a31-4df0-9e38-da10c2c717cc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:18.036658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f34376-2a31-4df0-9e38-da10c2c717cc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:18.036764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f34376-2a31-4df0-9e38-da10c2c717cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:18.042243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:18.056671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-61f34376-2a31-4df0-9e38-da10c2c717cc None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:18.057846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-61f34376-2a31-4df0-9e38-da10c2c717cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:18.127286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 622/1248] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:18.152344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c67c3-243b-45f2-9f5f-31c9459cdfd6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:18.171614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:18.270666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c67c3-243b-45f2-9f5f-31c9459cdfd6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:18.307359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d58e4c-c43e-45f3-8f09-521a32a9de64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:18.308181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d58e4c-c43e-45f3-8f09-521a32a9de64 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:18.308437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d58e4c-c43e-45f3-8f09-521a32a9de64 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:18.308734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d58e4c-c43e-45f3-8f09-521a32a9de64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:18.320173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d58e4c-c43e-45f3-8f09-521a32a9de64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-92d58e4c-c43e-45f3-8f09-521a32a9de64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:54:18.329102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:18.330383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:18.331516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 627/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:54:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:18.336376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f34376-2a31-4df0-9e38-da10c2c717cc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:18.446403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f34376-2a31-4df0-9e38-da10c2c717cc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:18.486498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:18.493333 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09e8df-beb4-450b-86e2-922a973b7ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:18.493999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09e8df-beb4-450b-86e2-922a973b7ecb None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:18.494311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09e8df-beb4-450b-86e2-922a973b7ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:18.494641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09e8df-beb4-450b-86e2-922a973b7ecb None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:18.495441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09e8df-beb4-450b-86e2-922a973b7ecb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:18.495749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09e8df-beb4-450b-86e2-922a973b7ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:18.497200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09e8df-beb4-450b-86e2-922a973b7ecb None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:18.506162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 623/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:54:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:18.512186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d58e4c-c43e-45f3-8f09-521a32a9de64 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:18.596634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:18.623475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d58e4c-c43e-45f3-8f09-521a32a9de64 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:18.671529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1e8a9-7f6f-4e94-ad35-b4969e29ac05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:18.672921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1e8a9-7f6f-4e94-ad35-b4969e29ac05 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:18.673210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1e8a9-7f6f-4e94-ad35-b4969e29ac05 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:18.673484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1e8a9-7f6f-4e94-ad35-b4969e29ac05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:18.685130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1e8a9-7f6f-4e94-ad35-b4969e29ac05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:18.686714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f1e8a9-7f6f-4e94-ad35-b4969e29ac05 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-a8f1e8a9-7f6f-4e94-ad35-b4969e29ac05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:18.705441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:18.706748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:18.707801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:18.707801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:18.707801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:18.707801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:18.707801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 628/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:18.713661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09e8df-beb4-450b-86e2-922a973b7ecb None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:18.831003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09e8df-beb4-450b-86e2-922a973b7ecb None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:18.868848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc21ba-6b4c-49f1-92f4-65b0383c9a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:18.869580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc21ba-6b4c-49f1-92f4-65b0383c9a54 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:18.869869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc21ba-6b4c-49f1-92f4-65b0383c9a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:18.870302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc21ba-6b4c-49f1-92f4-65b0383c9a54 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:18.871101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc21ba-6b4c-49f1-92f4-65b0383c9a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:18.871480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc21ba-6b4c-49f1-92f4-65b0383c9a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:18.873031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc21ba-6b4c-49f1-92f4-65b0383c9a54 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:18.886770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 624/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:18.893330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1e8a9-7f6f-4e94-ad35-b4969e29ac05 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:19.009123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1e8a9-7f6f-4e94-ad35-b4969e29ac05 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:19.040337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:19.050114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721126c-509d-4925-a05e-38087ca2aa2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:19.051139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721126c-509d-4925-a05e-38087ca2aa2c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:19.051461 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721126c-509d-4925-a05e-38087ca2aa2c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:19.051818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721126c-509d-4925-a05e-38087ca2aa2c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:19.063280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721126c-509d-4925-a05e-38087ca2aa2c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:19.065037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721126c-509d-4925-a05e-38087ca2aa2c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8721126c-509d-4925-a05e-38087ca2aa2c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:54:19.080740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:19.082057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:19.083311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 629/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:19.088112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc21ba-6b4c-49f1-92f4-65b0383c9a54 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:19.154319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:19.202883 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc21ba-6b4c-49f1-92f4-65b0383c9a54 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:19.244734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a504bba-be88-43fa-9b51-fb83a19ab27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:19.245427 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a504bba-be88-43fa-9b51-fb83a19ab27d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:19.245744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a504bba-be88-43fa-9b51-fb83a19ab27d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:19.246001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a504bba-be88-43fa-9b51-fb83a19ab27d None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:19.246894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a504bba-be88-43fa-9b51-fb83a19ab27d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:19.247208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a504bba-be88-43fa-9b51-fb83a19ab27d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:19.248703 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a504bba-be88-43fa-9b51-fb83a19ab27d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:19.269431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 625/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 26 20:54:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:19.275323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721126c-509d-4925-a05e-38087ca2aa2c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:19.380755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721126c-509d-4925-a05e-38087ca2aa2c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:19.417009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a99714-f851-43e9-b479-671d0aca4cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:19.417665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a99714-f851-43e9-b479-671d0aca4cde None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:19.417945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a99714-f851-43e9-b479-671d0aca4cde None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:19.418219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a99714-f851-43e9-b479-671d0aca4cde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:19.476507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:19.491458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a99714-f851-43e9-b479-671d0aca4cde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:19.492936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a99714-f851-43e9-b479-671d0aca4cde None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:19.602019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 630/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:54:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:54:19.608782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a504bba-be88-43fa-9b51-fb83a19ab27d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:19.719924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a504bba-be88-43fa-9b51-fb83a19ab27d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:19.733584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:19.758808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df82d72-c90e-4218-89c2-29a7e4cfb262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:19.759604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df82d72-c90e-4218-89c2-29a7e4cfb262 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:19.759893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df82d72-c90e-4218-89c2-29a7e4cfb262 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:19.760174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df82d72-c90e-4218-89c2-29a7e4cfb262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:19.770606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df82d72-c90e-4218-89c2-29a7e4cfb262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:19.772077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df82d72-c90e-4218-89c2-29a7e4cfb262 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-5df82d72-c90e-4218-89c2-29a7e4cfb262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:19.789709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:19.790838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:19.791913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:19.791913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:19.791913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:19.791913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:19.791913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 626/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:19.797173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a99714-f851-43e9-b479-671d0aca4cde None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:19.885958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:19.899848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a99714-f851-43e9-b479-671d0aca4cde None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:19.935052 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b065b9a-5fd9-45ae-bdf1-924e102cee3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:19.935705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b065b9a-5fd9-45ae-bdf1-924e102cee3d None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:19.935985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b065b9a-5fd9-45ae-bdf1-924e102cee3d None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:19.936260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b065b9a-5fd9-45ae-bdf1-924e102cee3d None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:19.936965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b065b9a-5fd9-45ae-bdf1-924e102cee3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:19.937291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b065b9a-5fd9-45ae-bdf1-924e102cee3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:19.939186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b065b9a-5fd9-45ae-bdf1-924e102cee3d None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:19.949839 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 631/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:19.955436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df82d72-c90e-4218-89c2-29a7e4cfb262 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:20.051538 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df82d72-c90e-4218-89c2-29a7e4cfb262 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:20.084276 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0645b-8cd7-4369-962f-e9a3d545ac8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:20.085045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0645b-8cd7-4369-962f-e9a3d545ac8e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:20.085327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0645b-8cd7-4369-962f-e9a3d545ac8e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:20.085599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0645b-8cd7-4369-962f-e9a3d545ac8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:20.091997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0645b-8cd7-4369-962f-e9a3d545ac8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:20.093445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0645b-8cd7-4369-962f-e9a3d545ac8e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-51b0645b-8cd7-4369-962f-e9a3d545ac8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:54:20.107415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:20.108481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:20.110214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:54:20.110214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:54:20.110214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:54:20.110214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:54:20.110214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:54:20.110214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:54:20.110214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:20.110214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:20.110214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 627/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:54:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:20.114898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b065b9a-5fd9-45ae-bdf1-924e102cee3d None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:20.211122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b065b9a-5fd9-45ae-bdf1-924e102cee3d None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:20.244133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221d21b-d341-4444-94a3-fc19ec790519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:20.244782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221d21b-d341-4444-94a3-fc19ec790519 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:20.245075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221d21b-d341-4444-94a3-fc19ec790519 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:20.245354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221d21b-d341-4444-94a3-fc19ec790519 None admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:20.246128 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221d21b-d341-4444-94a3-fc19ec790519 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:20.246438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221d21b-d341-4444-94a3-fc19ec790519 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:20.247882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221d21b-d341-4444-94a3-fc19ec790519 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:20.259667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 632/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:54:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:20.265941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0645b-8cd7-4369-962f-e9a3d545ac8e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:20.290859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:20.362455 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0645b-8cd7-4369-962f-e9a3d545ac8e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:20.394236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cda2e-874c-435f-b554-7f4abf4b760d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:20.394892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cda2e-874c-435f-b554-7f4abf4b760d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:20.395176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cda2e-874c-435f-b554-7f4abf4b760d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:20.395451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cda2e-874c-435f-b554-7f4abf4b760d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:20.396258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cda2e-874c-435f-b554-7f4abf4b760d None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=4edb53dc871047ce89ebb3b90143a6de) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:20.396567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cda2e-874c-435f-b554-7f4abf4b760d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:20.397989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cda2e-874c-435f-b554-7f4abf4b760d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:20.403178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 628/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:54:20 2022] GET /identity/v3/role_assignments?user.id=4edb53dc871047ce89ebb3b90143a6de&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:22.534904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aaa33-c782-4f04-ad90-e671d9d863fe None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:22.535465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aaa33-c782-4f04-ad90-e671d9d863fe None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:22.535465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120aaa33-c782-4f04-ad90-e671d9d863fe None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:22.536825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 633/1261] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:54:22.542040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712acf3-5d17-4376-9526-9dd6b0453e46 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:22.542492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712acf3-5d17-4376-9526-9dd6b0453e46 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:22.542619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712acf3-5d17-4376-9526-9dd6b0453e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:22.559458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6712acf3-5d17-4376-9526-9dd6b0453e46 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:22.560632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6712acf3-5d17-4376-9526-9dd6b0453e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:22.639155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 629/1262] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:22.664208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120aaa33-c782-4f04-ad90-e671d9d863fe None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:22.677581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:22.771172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120aaa33-c782-4f04-ad90-e671d9d863fe None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:22.805958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5870990-3d82-4962-a89f-892664df23a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:22.806770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5870990-3d82-4962-a89f-892664df23a5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:22.807053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5870990-3d82-4962-a89f-892664df23a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:22.807326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5870990-3d82-4962-a89f-892664df23a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:22.818387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5870990-3d82-4962-a89f-892664df23a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-e5870990-3d82-4962-a89f-892664df23a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:54:22.825980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:22.827013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:22.827867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 634/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:54:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:22.833271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712acf3-5d17-4376-9526-9dd6b0453e46 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:22.851128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:22.929834 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712acf3-5d17-4376-9526-9dd6b0453e46 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:22.962821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fbe35-34d6-45b1-865c-b5f5de619822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:22.963470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fbe35-34d6-45b1-865c-b5f5de619822 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:22.963757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fbe35-34d6-45b1-865c-b5f5de619822 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:22.964033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fbe35-34d6-45b1-865c-b5f5de619822 None admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:22.964775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fbe35-34d6-45b1-865c-b5f5de619822 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:22.965074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fbe35-34d6-45b1-865c-b5f5de619822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:22.966478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fbe35-34d6-45b1-865c-b5f5de619822 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:22.974793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 630/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:54:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:22.980932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5870990-3d82-4962-a89f-892664df23a5 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:23.044779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:23.077141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5870990-3d82-4962-a89f-892664df23a5 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:23.108739 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd9801-52d6-42e7-9230-d1546012e22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:23.110109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd9801-52d6-42e7-9230-d1546012e22e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:23.110389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd9801-52d6-42e7-9230-d1546012e22e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:23.110688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd9801-52d6-42e7-9230-d1546012e22e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:23.120705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd9801-52d6-42e7-9230-d1546012e22e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:23.122133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd9801-52d6-42e7-9230-d1546012e22e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-27bd9801-52d6-42e7-9230-d1546012e22e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:23.138741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:23.139773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:23.140892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:23.140892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:23.140892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:23.140892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:23.140892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 635/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:23.145547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fbe35-34d6-45b1-865c-b5f5de619822 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:23.223595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:23.242530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fbe35-34d6-45b1-865c-b5f5de619822 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:23.274084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f977d-3bd8-4655-bf79-06c2574218a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:23.274755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f977d-3bd8-4655-bf79-06c2574218a5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:23.275040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f977d-3bd8-4655-bf79-06c2574218a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:23.275317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f977d-3bd8-4655-bf79-06c2574218a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:23.276024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f977d-3bd8-4655-bf79-06c2574218a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:23.276329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f977d-3bd8-4655-bf79-06c2574218a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:23.277700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f977d-3bd8-4655-bf79-06c2574218a5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:23.289294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 631/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:23.294803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd9801-52d6-42e7-9230-d1546012e22e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:23.391009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd9801-52d6-42e7-9230-d1546012e22e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:23.409860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:23.423683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0c07b-8f90-4f4f-814f-fcc7f53c76e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:23.424460 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0c07b-8f90-4f4f-814f-fcc7f53c76e3 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:23.424741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0c07b-8f90-4f4f-814f-fcc7f53c76e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:23.425016 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0c07b-8f90-4f4f-814f-fcc7f53c76e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:23.432152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0c07b-8f90-4f4f-814f-fcc7f53c76e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:23.433563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb0c07b-8f90-4f4f-814f-fcc7f53c76e3 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-8cb0c07b-8f90-4f4f-814f-fcc7f53c76e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:54:23.447447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:23.448371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:23.449368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 636/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:23.454606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f977d-3bd8-4655-bf79-06c2574218a5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:23.550149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f977d-3bd8-4655-bf79-06c2574218a5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:23.584857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e3fd6-1850-4572-9ef7-c1be3deac2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:23.585646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e3fd6-1850-4572-9ef7-c1be3deac2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:23.585936 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e3fd6-1850-4572-9ef7-c1be3deac2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:23.586208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e3fd6-1850-4572-9ef7-c1be3deac2e0 None admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:23.587151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e3fd6-1850-4572-9ef7-c1be3deac2e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:23.587600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e3fd6-1850-4572-9ef7-c1be3deac2e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:23.589242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e3fd6-1850-4572-9ef7-c1be3deac2e0 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:23.600362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:23.612299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 632/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 26 20:54:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:23.618054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0c07b-8f90-4f4f-814f-fcc7f53c76e3 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:23.715339 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0c07b-8f90-4f4f-814f-fcc7f53c76e3 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:23.746790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191a7c5-8746-433b-b176-dcbbaff5083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:23.747574 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191a7c5-8746-433b-b176-dcbbaff5083c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:23.747860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191a7c5-8746-433b-b176-dcbbaff5083c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:23.748138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191a7c5-8746-433b-b176-dcbbaff5083c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:23.759018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191a7c5-8746-433b-b176-dcbbaff5083c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:23.760449 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191a7c5-8746-433b-b176-dcbbaff5083c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-1191a7c5-8746-433b-b176-dcbbaff5083c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:23.777666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:23.779063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:23.780334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:23.780334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:23.780334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:23.780334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:23.780334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 637/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:23.784570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e3fd6-1850-4572-9ef7-c1be3deac2e0 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:23.884581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e3fd6-1850-4572-9ef7-c1be3deac2e0 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:23.917193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2e3f6-cb49-4efd-a521-21091c8cdf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:23.917846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2e3f6-cb49-4efd-a521-21091c8cdf45 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:23.918131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2e3f6-cb49-4efd-a521-21091c8cdf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:23.918406 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2e3f6-cb49-4efd-a521-21091c8cdf45 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:23.919159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2e3f6-cb49-4efd-a521-21091c8cdf45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:23.919473 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2e3f6-cb49-4efd-a521-21091c8cdf45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:23.920849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2e3f6-cb49-4efd-a521-21091c8cdf45 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:23.931697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 633/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:23.937818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191a7c5-8746-433b-b176-dcbbaff5083c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:23.968754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:24.036278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191a7c5-8746-433b-b176-dcbbaff5083c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:24.068996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcd036-75b4-4c66-a7d5-96382109721f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:24.069777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd036-75b4-4c66-a7d5-96382109721f None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:24.070065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd036-75b4-4c66-a7d5-96382109721f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:24.070344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcd036-75b4-4c66-a7d5-96382109721f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:24.076821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcd036-75b4-4c66-a7d5-96382109721f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:24.078376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcd036-75b4-4c66-a7d5-96382109721f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-cbbcd036-75b4-4c66-a7d5-96382109721f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:54:24.092335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:24.093457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:24.094640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:54:24.094640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:54:24.094640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:54:24.094640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:54:24.094640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:54:24.094640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:54:24.094640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:24.094640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:24.094640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 638/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:24.099282 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2e3f6-cb49-4efd-a521-21091c8cdf45 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:24.159692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:24.200866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2e3f6-cb49-4efd-a521-21091c8cdf45 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:24.235381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12908dc-e627-4765-8f0e-04556b11ad3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:24.235971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12908dc-e627-4765-8f0e-04556b11ad3a None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:24.236258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12908dc-e627-4765-8f0e-04556b11ad3a None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:24.236538 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12908dc-e627-4765-8f0e-04556b11ad3a None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:24.237314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12908dc-e627-4765-8f0e-04556b11ad3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:24.237626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12908dc-e627-4765-8f0e-04556b11ad3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:24.239129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12908dc-e627-4765-8f0e-04556b11ad3a None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:24.251229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 634/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:54:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:24.257633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcd036-75b4-4c66-a7d5-96382109721f None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:24.343479 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:24.352455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcd036-75b4-4c66-a7d5-96382109721f None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:24.385358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23c75d-8117-4b3b-b869-cde0cccbb5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:24.386215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23c75d-8117-4b3b-b869-cde0cccbb5e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:24.386522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23c75d-8117-4b3b-b869-cde0cccbb5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:24.386802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23c75d-8117-4b3b-b869-cde0cccbb5e0 None admin] PATH_INFO: `/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/4edb53dc871047ce89ebb3b90143a6de/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:24.388862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23c75d-8117-4b3b-b869-cde0cccbb5e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ebd2318798e41a3b42eacbc8fb55499, user_id=4edb53dc871047ce89ebb3b90143a6de, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:24.390309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23c75d-8117-4b3b-b869-cde0cccbb5e0 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:24.407067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 639/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Thu May 26 20:54:24 2022] PUT /identity/v3/projects/9ebd2318798e41a3b42eacbc8fb55499/users/4edb53dc871047ce89ebb3b90143a6de/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 20:54:26.569630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f951b3-de93-418d-b321-2687ea467904 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:26.570150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f951b3-de93-418d-b321-2687ea467904 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:26.570276 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f951b3-de93-418d-b321-2687ea467904 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:26.571678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 635/1274] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:54:26.577184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bda2e-85d2-4927-93e9-fe90b1a87616 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:26.578103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bda2e-85d2-4927-93e9-fe90b1a87616 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:26.578280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bda2e-85d2-4927-93e9-fe90b1a87616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:26.598831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-fa2bda2e-85d2-4927-93e9-fe90b1a87616 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:26.600639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fa2bda2e-85d2-4927-93e9-fe90b1a87616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:26.667438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 640/1275] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:26.690941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f951b3-de93-418d-b321-2687ea467904 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:26.792503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f951b3-de93-418d-b321-2687ea467904 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:26.808819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:26.826198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda1f8f-3f23-4a87-a006-e1a7ab46c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:26.826964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda1f8f-3f23-4a87-a006-e1a7ab46c742 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:26.827247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda1f8f-3f23-4a87-a006-e1a7ab46c742 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:26.827521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda1f8f-3f23-4a87-a006-e1a7ab46c742 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:26.837859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda1f8f-3f23-4a87-a006-e1a7ab46c742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-1bda1f8f-3f23-4a87-a006-e1a7ab46c742 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 20:54:26.845509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:26.846668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:26.847899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 636/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Thu May 26 20:54:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:26.852597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bda2e-85d2-4927-93e9-fe90b1a87616 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:26.930429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:26.949686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bda2e-85d2-4927-93e9-fe90b1a87616 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:26.982144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd7342-a23b-4b04-bf88-8daab065d627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:26.982828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd7342-a23b-4b04-bf88-8daab065d627 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:26.983116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd7342-a23b-4b04-bf88-8daab065d627 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:26.983395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd7342-a23b-4b04-bf88-8daab065d627 None admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:26.984132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd7342-a23b-4b04-bf88-8daab065d627 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:26.984446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd7342-a23b-4b04-bf88-8daab065d627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:26.985804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd7342-a23b-4b04-bf88-8daab065d627 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:26.993557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 641/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Thu May 26 20:54:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:26.999059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda1f8f-3f23-4a87-a006-e1a7ab46c742 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:27.096135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda1f8f-3f23-4a87-a006-e1a7ab46c742 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:27.128625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a9844-df3d-4b6d-8b94-e8240359cf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:27.129392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a9844-df3d-4b6d-8b94-e8240359cf98 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:27.129676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a9844-df3d-4b6d-8b94-e8240359cf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:27.129953 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a9844-df3d-4b6d-8b94-e8240359cf98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:27.140278 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a9844-df3d-4b6d-8b94-e8240359cf98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:27.141721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a9844-df3d-4b6d-8b94-e8240359cf98 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-c59a9844-df3d-4b6d-8b94-e8240359cf98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:27.158283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:27.159352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:27.160466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:27.160466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:27.160466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:27.160466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:27.160466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 637/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:27.165052 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd7342-a23b-4b04-bf88-8daab065d627 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:27.261071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd7342-a23b-4b04-bf88-8daab065d627 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:27.292378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababfd37-7465-4080-8fd4-17fb72af2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:27.293018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababfd37-7465-4080-8fd4-17fb72af2fbe None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:27.293304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababfd37-7465-4080-8fd4-17fb72af2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:27.293584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababfd37-7465-4080-8fd4-17fb72af2fbe None admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:27.294294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababfd37-7465-4080-8fd4-17fb72af2fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:27.294631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababfd37-7465-4080-8fd4-17fb72af2fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:27.295980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababfd37-7465-4080-8fd4-17fb72af2fbe None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:27.307672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 642/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:27.313419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a9844-df3d-4b6d-8b94-e8240359cf98 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:27.342927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:27.412296 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a9844-df3d-4b6d-8b94-e8240359cf98 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:27.444955 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f54c1b-b8c4-47a9-ac52-5f933bd10c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:27.445733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f54c1b-b8c4-47a9-ac52-5f933bd10c1e None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:27.446017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f54c1b-b8c4-47a9-ac52-5f933bd10c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:27.446295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f54c1b-b8c4-47a9-ac52-5f933bd10c1e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:27.453498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f54c1b-b8c4-47a9-ac52-5f933bd10c1e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:27.454987 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f54c1b-b8c4-47a9-ac52-5f933bd10c1e None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-44f54c1b-b8c4-47a9-ac52-5f933bd10c1e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 20:54:27.469400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:27.470491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:27.471468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 638/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:27.476187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababfd37-7465-4080-8fd4-17fb72af2fbe None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:27.479890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:27.574649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababfd37-7465-4080-8fd4-17fb72af2fbe None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:27.617202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b16fe-f403-4e27-a4cb-dcc18e543ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:27.617864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b16fe-f403-4e27-a4cb-dcc18e543ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:27.618145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b16fe-f403-4e27-a4cb-dcc18e543ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:27.618417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b16fe-f403-4e27-a4cb-dcc18e543ff2 None admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:27.619269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b16fe-f403-4e27-a4cb-dcc18e543ff2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:27.619599 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b16fe-f403-4e27-a4cb-dcc18e543ff2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:27.621015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b16fe-f403-4e27-a4cb-dcc18e543ff2 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:27.638170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 643/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Thu May 26 20:54:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:27.644441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f54c1b-b8c4-47a9-ac52-5f933bd10c1e None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:27.734474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:27.751141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f54c1b-b8c4-47a9-ac52-5f933bd10c1e None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:27.784298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9ad40-a753-4827-8548-32fefe17f58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:27.784934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ad40-a753-4827-8548-32fefe17f58f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:27.785214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ad40-a753-4827-8548-32fefe17f58f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:27.785490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9ad40-a753-4827-8548-32fefe17f58f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:27.850233 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9ad40-a753-4827-8548-32fefe17f58f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:27.851727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9ad40-a753-4827-8548-32fefe17f58f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:27.934839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:27.945524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 639/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Thu May 26 20:54:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 20:54:27.951325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b16fe-f403-4e27-a4cb-dcc18e543ff2 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:28.010174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:28.053669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b16fe-f403-4e27-a4cb-dcc18e543ff2 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:28.088547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198dfe-ccb8-4200-8eff-f8587659fb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:28.089336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198dfe-ccb8-4200-8eff-f8587659fb36 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:28.089620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198dfe-ccb8-4200-8eff-f8587659fb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:28.089896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96198dfe-ccb8-4200-8eff-f8587659fb36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:28.099776 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96198dfe-ccb8-4200-8eff-f8587659fb36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:28.101259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96198dfe-ccb8-4200-8eff-f8587659fb36 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-96198dfe-ccb8-4200-8eff-f8587659fb36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 20:54:28.121457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:28.122549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:28.123697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 20:54:28.123697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 20:54:28.123697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 20:54:28.123697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:28.123697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 644/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:28.129135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9ad40-a753-4827-8548-32fefe17f58f None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:28.222714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9ad40-a753-4827-8548-32fefe17f58f None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:28.254967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44596bcb-f81b-4dcf-bd78-8292953f4502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:28.255871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44596bcb-f81b-4dcf-bd78-8292953f4502 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:28.256156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44596bcb-f81b-4dcf-bd78-8292953f4502 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:28.256432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44596bcb-f81b-4dcf-bd78-8292953f4502 None admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:28.257137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44596bcb-f81b-4dcf-bd78-8292953f4502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:28.257440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44596bcb-f81b-4dcf-bd78-8292953f4502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:28.258830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44596bcb-f81b-4dcf-bd78-8292953f4502 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:28.269451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 640/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Thu May 26 20:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:28.274893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198dfe-ccb8-4200-8eff-f8587659fb36 None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:28.379687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96198dfe-ccb8-4200-8eff-f8587659fb36 None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:28.394075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:28.414247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fc582-93a4-4da2-bd76-73a349a07b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:28.415070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fc582-93a4-4da2-bd76-73a349a07b6e None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:28.415358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fc582-93a4-4da2-bd76-73a349a07b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:28.415636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fc582-93a4-4da2-bd76-73a349a07b6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:28.422754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fc582-93a4-4da2-bd76-73a349a07b6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:28.424190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fc582-93a4-4da2-bd76-73a349a07b6e None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application [None req-2a6fc582-93a4-4da2-bd76-73a349a07b6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 20:54:28.437610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application with Lock( May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:28.438696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:28.439638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 20:54:28.439638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 20:54:28.439638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 20:54:28.439638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 20:54:28.439638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 20:54:28.439638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 20:54:28.439638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 20:54:28.439638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: ERROR keystone.server.flask.application May 26 20:54:28.439638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 645/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Thu May 26 20:54:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:28.444695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44596bcb-f81b-4dcf-bd78-8292953f4502 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:28.492594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:28.541495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44596bcb-f81b-4dcf-bd78-8292953f4502 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:28.573723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a2c4f1-c3ce-4410-8b34-5c4fea7fa46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:28.574508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2c4f1-c3ce-4410-8b34-5c4fea7fa46f None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:28.574810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2c4f1-c3ce-4410-8b34-5c4fea7fa46f None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:28.575214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a2c4f1-c3ce-4410-8b34-5c4fea7fa46f None admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:28.576315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2c4f1-c3ce-4410-8b34-5c4fea7fa46f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:28.576502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2c4f1-c3ce-4410-8b34-5c4fea7fa46f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:28.578761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a2c4f1-c3ce-4410-8b34-5c4fea7fa46f None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:28.592894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 641/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Thu May 26 20:54:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:28.600232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fc582-93a4-4da2-bd76-73a349a07b6e None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:28.696762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fc582-93a4-4da2-bd76-73a349a07b6e None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:28.729571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4835f4-9f77-477f-85cf-366be48be541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:28.730206 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4835f4-9f77-477f-85cf-366be48be541 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:28.730523 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4835f4-9f77-477f-85cf-366be48be541 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:28.730805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4835f4-9f77-477f-85cf-366be48be541 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:28.731622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4835f4-9f77-477f-85cf-366be48be541 None admin] RBAC: Adding query filter params (role.id=b8ccbdae4af04d82bdc9bbba141a6b8d, scope.project.id=9ebd2318798e41a3b42eacbc8fb55499, user.id=4edb53dc871047ce89ebb3b90143a6de) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:28.731934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4835f4-9f77-477f-85cf-366be48be541 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:28.733348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4835f4-9f77-477f-85cf-366be48be541 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:28.739085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 646/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Thu May 26 20:54:28 2022] GET /identity/v3/role_assignments?user.id=4edb53dc871047ce89ebb3b90143a6de&scope.project.id=9ebd2318798e41a3b42eacbc8fb55499&role.id=b8ccbdae4af04d82bdc9bbba141a6b8d => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:30.908471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d764565-bf1b-4a49-9885-d2454645276e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:30.908946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d764565-bf1b-4a49-9885-d2454645276e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:30.909037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d764565-bf1b-4a49-9885-d2454645276e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:30.910377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 642/1288] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:54:30.915646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ea851-c175-493b-97e6-ef214f9112b6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:30.916042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ea851-c175-493b-97e6-ef214f9112b6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:30.916203 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ea851-c175-493b-97e6-ef214f9112b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:30.932995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-dc6ea851-c175-493b-97e6-ef214f9112b6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:30.934161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dc6ea851-c175-493b-97e6-ef214f9112b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:30.962706 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:30.998288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 647/1289] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:31.020294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d764565-bf1b-4a49-9885-d2454645276e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:31.118234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d764565-bf1b-4a49-9885-d2454645276e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:31.128839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:31.153181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36271f-ff0e-4c19-ace0-1dedfc74d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:31.153966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36271f-ff0e-4c19-ace0-1dedfc74d39d None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:31.154250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36271f-ff0e-4c19-ace0-1dedfc74d39d None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:31.154572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36271f-ff0e-4c19-ace0-1dedfc74d39d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:31.155220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36271f-ff0e-4c19-ace0-1dedfc74d39d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:31.156633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36271f-ff0e-4c19-ace0-1dedfc74d39d None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-4f36271f-ff0e-4c19-ace0-1dedfc74d39d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:31.168274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:31.169481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:31.170574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 643/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 26 20:54:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:31.176044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ea851-c175-493b-97e6-ef214f9112b6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:31.271224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ea851-c175-493b-97e6-ef214f9112b6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:31.304737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39acad6-ce52-48b6-8d3c-f5e24a956408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:31.305390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39acad6-ce52-48b6-8d3c-f5e24a956408 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:31.305676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39acad6-ce52-48b6-8d3c-f5e24a956408 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:31.305956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39acad6-ce52-48b6-8d3c-f5e24a956408 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:31.306682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39acad6-ce52-48b6-8d3c-f5e24a956408 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:31.306990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39acad6-ce52-48b6-8d3c-f5e24a956408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:31.308349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39acad6-ce52-48b6-8d3c-f5e24a956408 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:31.313041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 648/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 26 20:54:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:31.318428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36271f-ff0e-4c19-ace0-1dedfc74d39d None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:31.413373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36271f-ff0e-4c19-ace0-1dedfc74d39d None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:31.451839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b54cd6-05ec-44da-b2e5-3ee5016354bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:31.452514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b54cd6-05ec-44da-b2e5-3ee5016354bf None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:31.452796 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b54cd6-05ec-44da-b2e5-3ee5016354bf None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:31.453071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b54cd6-05ec-44da-b2e5-3ee5016354bf None admin] PATH_INFO: `/v3/services` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:31.453769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b54cd6-05ec-44da-b2e5-3ee5016354bf None admin] RBAC: Adding query filter params (type=placement) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:31.454074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b54cd6-05ec-44da-b2e5-3ee5016354bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:31.455482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b54cd6-05ec-44da-b2e5-3ee5016354bf None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:31.459841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 644/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 26 20:54:31 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:33.626370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b38e60-26f1-4997-b9f2-963a84b8c6ad None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:33.627322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b38e60-26f1-4997-b9f2-963a84b8c6ad None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:33.627322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b38e60-26f1-4997-b9f2-963a84b8c6ad None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:33.628319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 649/1293] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:54:33.634131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b3b83-acc9-4c9a-9e06-7c42a2851645 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:33.634606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b3b83-acc9-4c9a-9e06-7c42a2851645 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:33.634824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b3b83-acc9-4c9a-9e06-7c42a2851645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:33.653941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9f3b3b83-acc9-4c9a-9e06-7c42a2851645 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:33.655176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9f3b3b83-acc9-4c9a-9e06-7c42a2851645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:33.685395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:33.721943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 645/1294] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:33.744865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b38e60-26f1-4997-b9f2-963a84b8c6ad None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:33.774763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:33.844650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b38e60-26f1-4997-b9f2-963a84b8c6ad None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:33.876994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a643720-b13d-4ec1-ab59-87d8a28db21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:33.877594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a643720-b13d-4ec1-ab59-87d8a28db21f None admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:33.877877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a643720-b13d-4ec1-ab59-87d8a28db21f None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:33.878154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a643720-b13d-4ec1-ab59-87d8a28db21f None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:33.878912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a643720-b13d-4ec1-ab59-87d8a28db21f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:33.880272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a643720-b13d-4ec1-ab59-87d8a28db21f None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:33.893312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 650/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Thu May 26 20:54:33 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:54:36.034335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302ad5-9b68-4e38-9919-fe824d1422a7 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:36.034883 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302ad5-9b68-4e38-9919-fe824d1422a7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:36.034969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302ad5-9b68-4e38-9919-fe824d1422a7 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:36.036287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 646/1296] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:54:36.041304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0bc7e-9860-4c8f-9432-469f08542af1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:36.041714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0bc7e-9860-4c8f-9432-469f08542af1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:36.041866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0bc7e-9860-4c8f-9432-469f08542af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:36.058831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7bc0bc7e-9860-4c8f-9432-469f08542af1 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:36.060025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7bc0bc7e-9860-4c8f-9432-469f08542af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:36.129029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 651/1297] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:36.156868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302ad5-9b68-4e38-9919-fe824d1422a7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:36.253083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302ad5-9b68-4e38-9919-fe824d1422a7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:36.286420 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a9e90-5d86-4fbf-815f-8a18af5cb0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:36.287220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a9e90-5d86-4fbf-815f-8a18af5cb0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:36.287502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a9e90-5d86-4fbf-815f-8a18af5cb0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:36.287776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a9e90-5d86-4fbf-815f-8a18af5cb0e2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:36.288417 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a9e90-5d86-4fbf-815f-8a18af5cb0e2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:36.289788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a9e90-5d86-4fbf-815f-8a18af5cb0e2 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-0a1a9e90-5d86-4fbf-815f-8a18af5cb0e2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:36.301376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:36.302583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:36.303464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 647/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 26 20:54:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:36.308530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0bc7e-9860-4c8f-9432-469f08542af1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:36.326050 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:36.404965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0bc7e-9860-4c8f-9432-469f08542af1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:36.441374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c815c4f-03ae-4410-80cb-98a195dc9d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:36.442044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c815c4f-03ae-4410-80cb-98a195dc9d1c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:36.442327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c815c4f-03ae-4410-80cb-98a195dc9d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:36.442635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c815c4f-03ae-4410-80cb-98a195dc9d1c None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:36.443343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c815c4f-03ae-4410-80cb-98a195dc9d1c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:36.443654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c815c4f-03ae-4410-80cb-98a195dc9d1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:36.445022 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c815c4f-03ae-4410-80cb-98a195dc9d1c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:36.450109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 652/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 26 20:54:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:36.456652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a9e90-5d86-4fbf-815f-8a18af5cb0e2 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:36.487288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:36.553845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a9e90-5d86-4fbf-815f-8a18af5cb0e2 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:36.590623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc64cb-b19a-403b-b0ca-7d3e31e60847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:36.591527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc64cb-b19a-403b-b0ca-7d3e31e60847 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:36.591853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc64cb-b19a-403b-b0ca-7d3e31e60847 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:36.592251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc64cb-b19a-403b-b0ca-7d3e31e60847 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:36.593410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc64cb-b19a-403b-b0ca-7d3e31e60847 None admin] RBAC: Adding query filter params (interface=public, service_id=ecd1da0ac81f44e2818b0400619cc69f, region_id=RegionOne) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:36.593764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc64cb-b19a-403b-b0ca-7d3e31e60847 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:36.595551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc64cb-b19a-403b-b0ca-7d3e31e60847 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:36.601444 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 648/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Thu May 26 20:54:36 2022] GET /identity/v3/endpoints?service_id=ecd1da0ac81f44e2818b0400619cc69f&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:36.607166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c815c4f-03ae-4410-80cb-98a195dc9d1c None admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:36.699592 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:36.717670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c815c4f-03ae-4410-80cb-98a195dc9d1c None admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:36.750422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a213770d-1a95-4fe6-b0f0-dc897d48402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:36.751106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213770d-1a95-4fe6-b0f0-dc897d48402c None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:36.751396 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213770d-1a95-4fe6-b0f0-dc897d48402c None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:36.751675 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a213770d-1a95-4fe6-b0f0-dc897d48402c None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:36.752357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213770d-1a95-4fe6-b0f0-dc897d48402c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:36.753731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a213770d-1a95-4fe6-b0f0-dc897d48402c None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:36.762221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 653/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Thu May 26 20:54:36 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:54:38.937214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437c3d1-6530-4095-a42c-a3192a471292 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:38.937742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437c3d1-6530-4095-a42c-a3192a471292 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:38.937836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437c3d1-6530-4095-a42c-a3192a471292 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:38.939162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 649/1302] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:54:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:54:38.944309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ba68b-1e38-4d56-acd5-28f218e10428 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:38.944677 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ba68b-1e38-4d56-acd5-28f218e10428 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:38.944865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ba68b-1e38-4d56-acd5-28f218e10428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:38.961634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b36ba68b-1e38-4d56-acd5-28f218e10428 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:54:38.962839 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b36ba68b-1e38-4d56-acd5-28f218e10428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:54:39.026263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 654/1303] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:54:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 20:54:39.048562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437c3d1-6530-4095-a42c-a3192a471292 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:39.140989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:39.148118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437c3d1-6530-4095-a42c-a3192a471292 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:39.183392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2711fc-973c-4a49-97b5-7052b2820ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:39.184176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2711fc-973c-4a49-97b5-7052b2820ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:39.184459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2711fc-973c-4a49-97b5-7052b2820ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:39.184734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2711fc-973c-4a49-97b5-7052b2820ba5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:39.185377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2711fc-973c-4a49-97b5-7052b2820ba5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:39.187437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2711fc-973c-4a49-97b5-7052b2820ba5 None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application [None req-fc2711fc-973c-4a49-97b5-7052b2820ba5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 20:54:39.199053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 20:54:39.200216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.get_or_create( May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application with Lock( May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self._enter() May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application return self.creator() May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application created_value = creator( May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: ERROR keystone.server.flask.application May 26 20:54:39.201269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 650/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Thu May 26 20:54:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 20:54:39.206312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ba68b-1e38-4d56-acd5-28f218e10428 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:39.280186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:39.302796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ba68b-1e38-4d56-acd5-28f218e10428 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:39.335736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe3dd2-a6e5-40b2-bb42-ea18c6b801f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:39.336385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe3dd2-a6e5-40b2-bb42-ea18c6b801f5 None admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:39.336665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe3dd2-a6e5-40b2-bb42-ea18c6b801f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:39.336940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe3dd2-a6e5-40b2-bb42-ea18c6b801f5 None admin] PATH_INFO: `/v3/services` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:39.337630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe3dd2-a6e5-40b2-bb42-ea18c6b801f5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 20:54:39.337937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe3dd2-a6e5-40b2-bb42-ea18c6b801f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:39.339331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe3dd2-a6e5-40b2-bb42-ea18c6b801f5 None admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:39.344298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 655/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Thu May 26 20:54:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 20:54:39.349829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2711fc-973c-4a49-97b5-7052b2820ba5 None admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:54:39.446398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2711fc-973c-4a49-97b5-7052b2820ba5 None admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:54:39.481158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff55a04-f61d-40ac-9762-abf05f9fa3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:54:39.481804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55a04-f61d-40ac-9762-abf05f9fa3ef None admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:54:39.482089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55a04-f61d-40ac-9762-abf05f9fa3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:54:39.482367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55a04-f61d-40ac-9762-abf05f9fa3ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:54:39.483125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff55a04-f61d-40ac-9762-abf05f9fa3ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:54:39.485180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff55a04-f61d-40ac-9762-abf05f9fa3ef None admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:54:39.502589 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 651/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Thu May 26 20:54:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 20:54:49.341766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=30 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:54:49.498312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:42.552678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84508cc-888f-41dd-b038-2e5c101d480b None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:42.553717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84508cc-888f-41dd-b038-2e5c101d480b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:42.554016 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84508cc-888f-41dd-b038-2e5c101d480b None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:42.555662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 656/1307] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:55:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:55:42.561392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea00b9-3f48-4fea-a1a5-1efa38948673 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:42.561875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea00b9-3f48-4fea-a1a5-1efa38948673 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:42.561875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea00b9-3f48-4fea-a1a5-1efa38948673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:42.644421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-76ea00b9-3f48-4fea-a1a5-1efa38948673 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:55:42.645811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-76ea00b9-3f48-4fea-a1a5-1efa38948673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:42.896392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 652/1308] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:55:42.933682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7dfbb0-2645-49f9-b349-29d28ea164ce None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:42.934227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7dfbb0-2645-49f9-b349-29d28ea164ce None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:42.934563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7dfbb0-2645-49f9-b349-29d28ea164ce None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:42.936256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 657/1309] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 26 20:55:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:55:42.943099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d310a2-37e0-4eb8-8097-2e3a6f64baf4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:42.943484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d310a2-37e0-4eb8-8097-2e3a6f64baf4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:42.943840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d310a2-37e0-4eb8-8097-2e3a6f64baf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:43.036263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-97d310a2-37e0-4eb8-8097-2e3a6f64baf4 None None] MFA Rules not processed for user `94b8d8b089264b5192a651b41bf2a205`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:55:43.037487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-97d310a2-37e0-4eb8-8097-2e3a6f64baf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:43.061355 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:43.123839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 653/1310] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 26 20:55:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:55:43.133329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7dfbb0-2645-49f9-b349-29d28ea164ce None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:55:43.135184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2e7dfbb0-2645-49f9-b349-29d28ea164ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:43.259265 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7dfbb0-2645-49f9-b349-29d28ea164ce None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:55:43.335593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc38fd1-63c2-4627-a13a-6d9ef15186a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:55:43.336153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc38fd1-63c2-4627-a13a-6d9ef15186a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:43.336431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc38fd1-63c2-4627-a13a-6d9ef15186a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:43.336702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc38fd1-63c2-4627-a13a-6d9ef15186a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:43.338319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dbc38fd1-63c2-4627-a13a-6d9ef15186a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:43.393802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:43.450661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc38fd1-63c2-4627-a13a-6d9ef15186a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:55:43.452689 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc38fd1-63c2-4627-a13a-6d9ef15186a2 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:55:43.473736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 658/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:55:45.972127 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58ca33-078b-4ae5-a9c3-d8dd9e2c80e5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:45.972699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58ca33-078b-4ae5-a9c3-d8dd9e2c80e5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:45.972699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b58ca33-078b-4ae5-a9c3-d8dd9e2c80e5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:45.974057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 654/1312] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:55:45.979642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac8991-d7ac-4d05-9cb0-bf29718d3232 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:45.980091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac8991-d7ac-4d05-9cb0-bf29718d3232 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:45.980448 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac8991-d7ac-4d05-9cb0-bf29718d3232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:46.072452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8aac8991-d7ac-4d05-9cb0-bf29718d3232 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:55:46.074209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8aac8991-d7ac-4d05-9cb0-bf29718d3232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:46.197575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 659/1313] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:55:46.236513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58ca33-078b-4ae5-a9c3-d8dd9e2c80e5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:55:46.238046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5b58ca33-078b-4ae5-a9c3-d8dd9e2c80e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:46.356424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b58ca33-078b-4ae5-a9c3-d8dd9e2c80e5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:55:46.431688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:46.434469 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbeb420-0c5e-46d3-9f01-ebca2f523ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:55:46.435696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeb420-0c5e-46d3-9f01-ebca2f523ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:46.436040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeb420-0c5e-46d3-9f01-ebca2f523ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:46.436325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbeb420-0c5e-46d3-9f01-ebca2f523ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:46.438801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5cbeb420-0c5e-46d3-9f01-ebca2f523ffc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:46.556104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeb420-0c5e-46d3-9f01-ebca2f523ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:55:46.558048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbeb420-0c5e-46d3-9f01-ebca2f523ffc service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:55:46.580030 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 655/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:55:49.116158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82b990-10e1-48c5-976c-a3b8e74127f2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:49.116728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82b990-10e1-48c5-976c-a3b8e74127f2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:49.116728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82b990-10e1-48c5-976c-a3b8e74127f2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:49.117998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 660/1315] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:55:49.123065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b1fde-9087-4f65-ae70-d29a9f1a507a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:49.123505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b1fde-9087-4f65-ae70-d29a9f1a507a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:49.123634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b1fde-9087-4f65-ae70-d29a9f1a507a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:49.187614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-894b1fde-9087-4f65-ae70-d29a9f1a507a None None] MFA Rules not processed for user `9a041adb8e4f4e4abce713c554736b14`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:55:49.188811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-894b1fde-9087-4f65-ae70-d29a9f1a507a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:49.238669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:49.283051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 656/1316] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:55:49 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:55:49.321907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82b990-10e1-48c5-976c-a3b8e74127f2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:55:49.323477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4b82b990-10e1-48c5-976c-a3b8e74127f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:49.333585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:49.424688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82b990-10e1-48c5-976c-a3b8e74127f2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:55:49.495244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbf8c1-8ec6-43dc-8e55-3ee64419b696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:55:49.495885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbf8c1-8ec6-43dc-8e55-3ee64419b696 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:49.496168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbf8c1-8ec6-43dc-8e55-3ee64419b696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:49.496446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbf8c1-8ec6-43dc-8e55-3ee64419b696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:49.498621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e5dbf8c1-8ec6-43dc-8e55-3ee64419b696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:49.559305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:49.606039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbf8c1-8ec6-43dc-8e55-3ee64419b696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:55:49.608021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dbf8c1-8ec6-43dc-8e55-3ee64419b696 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:55:49.627331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 661/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:55:49 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:55:52.348128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb500f-83df-4328-9edf-a25ab9de76b2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:52.348692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb500f-83df-4328-9edf-a25ab9de76b2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:52.348692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb500f-83df-4328-9edf-a25ab9de76b2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:52.350080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 657/1318] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:55:52.355409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db75a1-d6c6-4107-a340-5676b3d535b8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:52.355829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db75a1-d6c6-4107-a340-5676b3d535b8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:52.355963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db75a1-d6c6-4107-a340-5676b3d535b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:52.418970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-39db75a1-d6c6-4107-a340-5676b3d535b8 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:55:52.420786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-39db75a1-d6c6-4107-a340-5676b3d535b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:52.515205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 662/1319] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:55:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:55:52.553684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb500f-83df-4328-9edf-a25ab9de76b2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:55:52.555228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-27cb500f-83df-4328-9edf-a25ab9de76b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:52.659829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb500f-83df-4328-9edf-a25ab9de76b2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:55:52.729541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63f835-5e76-4af7-8a15-1a59b0f3837d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:55:52.730182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63f835-5e76-4af7-8a15-1a59b0f3837d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:52.730485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63f835-5e76-4af7-8a15-1a59b0f3837d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:52.730769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63f835-5e76-4af7-8a15-1a59b0f3837d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:52.732428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dc63f835-5e76-4af7-8a15-1a59b0f3837d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:52.739610 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:52.838895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63f835-5e76-4af7-8a15-1a59b0f3837d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:55:52.840353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63f835-5e76-4af7-8a15-1a59b0f3837d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:55:52.858564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 658/1320] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:55:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:55:55.683111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb625b-5abf-4683-8d3c-d8ef313896da None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:55.683675 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb625b-5abf-4683-8d3c-d8ef313896da None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:55.683782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb625b-5abf-4683-8d3c-d8ef313896da None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:55.685158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 663/1321] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:55:55.690570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514155dd-c6db-40ec-88ee-2db40092271a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:55.690968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514155dd-c6db-40ec-88ee-2db40092271a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:55.691237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514155dd-c6db-40ec-88ee-2db40092271a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:55.755695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-514155dd-c6db-40ec-88ee-2db40092271a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:55:55.756893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-514155dd-c6db-40ec-88ee-2db40092271a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:55.818348 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:55.857881 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 659/1322] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:55:55.893316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb625b-5abf-4683-8d3c-d8ef313896da None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:55:55.894894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-16eb625b-5abf-4683-8d3c-d8ef313896da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:55.914254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:55.998392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb625b-5abf-4683-8d3c-d8ef313896da None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:55:56.071171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac284ae-f5b3-45bf-baac-3fa697e81e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:55:56.071831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac284ae-f5b3-45bf-baac-3fa697e81e18 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:56.072116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac284ae-f5b3-45bf-baac-3fa697e81e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:56.072395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac284ae-f5b3-45bf-baac-3fa697e81e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:56.074015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2ac284ae-f5b3-45bf-baac-3fa697e81e18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:56.144818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:56.181863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac284ae-f5b3-45bf-baac-3fa697e81e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:55:56.183329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac284ae-f5b3-45bf-baac-3fa697e81e18 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:55:56.202151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 664/1323] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:55:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:55:58.490824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b718504-bc0b-434a-80a9-a4a10c08fcf0 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:58.491377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b718504-bc0b-434a-80a9-a4a10c08fcf0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:58.491377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b718504-bc0b-434a-80a9-a4a10c08fcf0 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:58.492754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 660/1324] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:55:58.498064 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5c62b-29d7-48d9-9071-c1d4d2b93c3d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:58.498528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5c62b-29d7-48d9-9071-c1d4d2b93c3d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:58.498661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5c62b-29d7-48d9-9071-c1d4d2b93c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:58.563124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a4a5c62b-29d7-48d9-9071-c1d4d2b93c3d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:55:58.564341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a4a5c62b-29d7-48d9-9071-c1d4d2b93c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:58.666795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 665/1325] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:55:58.701821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b718504-bc0b-434a-80a9-a4a10c08fcf0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:55:58.703360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0b718504-bc0b-434a-80a9-a4a10c08fcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:58.804051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b718504-bc0b-434a-80a9-a4a10c08fcf0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:55:58.883314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad1ce1-5d64-4264-b9db-78bfc1204d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:55:58.883966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1ce1-5d64-4264-b9db-78bfc1204d26 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:55:58.884253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1ce1-5d64-4264-b9db-78bfc1204d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:55:58.884533 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1ce1-5d64-4264-b9db-78bfc1204d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:55:58.886161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-36ad1ce1-5d64-4264-b9db-78bfc1204d26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:55:58.905255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:55:59.001579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad1ce1-5d64-4264-b9db-78bfc1204d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:55:59.003080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad1ce1-5d64-4264-b9db-78bfc1204d26 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:55:59.022327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 661/1326] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:01.292578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edcdb4-8ea3-4ee1-8cff-571861e3ef73 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:01.293082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edcdb4-8ea3-4ee1-8cff-571861e3ef73 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:01.293082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edcdb4-8ea3-4ee1-8cff-571861e3ef73 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:01.295109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 666/1327] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:01.300215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc73f49-9e0a-414a-8705-59f380d44af5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:01.300620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc73f49-9e0a-414a-8705-59f380d44af5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:01.300782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc73f49-9e0a-414a-8705-59f380d44af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:01.366048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4fc73f49-9e0a-414a-8705-59f380d44af5 None None] MFA Rules not processed for user `9a041adb8e4f4e4abce713c554736b14`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:01.367301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4fc73f49-9e0a-414a-8705-59f380d44af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:01.432554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:01.464779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 662/1328] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:01 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:01.502231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edcdb4-8ea3-4ee1-8cff-571861e3ef73 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:01.503797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a4edcdb4-8ea3-4ee1-8cff-571861e3ef73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:01.535327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:01.609311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edcdb4-8ea3-4ee1-8cff-571861e3ef73 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:01.682470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1f4715-a64b-4742-8afc-d2ee96ff6269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:01.683123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f4715-a64b-4742-8afc-d2ee96ff6269 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:01.683409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f4715-a64b-4742-8afc-d2ee96ff6269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:01.683686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f4715-a64b-4742-8afc-d2ee96ff6269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:01.685876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8d1f4715-a64b-4742-8afc-d2ee96ff6269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:01.761858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:01.795008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f4715-a64b-4742-8afc-d2ee96ff6269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:01.796449 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1f4715-a64b-4742-8afc-d2ee96ff6269 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:01.817392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 667/1329] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:04.253353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ccdc5-df80-40d9-9dcd-e884e3847aa8 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:04.253881 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ccdc5-df80-40d9-9dcd-e884e3847aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:04.253999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ccdc5-df80-40d9-9dcd-e884e3847aa8 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:04.255550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 663/1330] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:04.260874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019b4cd-8ff6-4c91-9a4f-ef75b569f6e3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:04.261286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019b4cd-8ff6-4c91-9a4f-ef75b569f6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:04.261409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6019b4cd-8ff6-4c91-9a4f-ef75b569f6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:04.329554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6019b4cd-8ff6-4c91-9a4f-ef75b569f6e3 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:04.331008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6019b4cd-8ff6-4c91-9a4f-ef75b569f6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:04.429078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 668/1331] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:04.479811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482be6de-4b11-41a7-a54c-40b69bb7743a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:04.480101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482be6de-4b11-41a7-a54c-40b69bb7743a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:04.480385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482be6de-4b11-41a7-a54c-40b69bb7743a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:04.481780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 664/1332] 10.222.0.57 () {54 vars in 964 bytes} [Thu May 26 20:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:04.488941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510b03d-2f8e-40a5-a6f3-fe7a83fdda73 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:04.489240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510b03d-2f8e-40a5-a6f3-fe7a83fdda73 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:04.489527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510b03d-2f8e-40a5-a6f3-fe7a83fdda73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:04.536154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:04.570577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5510b03d-2f8e-40a5-a6f3-fe7a83fdda73 None None] MFA Rules not processed for user `94b8d8b089264b5192a651b41bf2a205`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:04.571793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5510b03d-2f8e-40a5-a6f3-fe7a83fdda73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:04.653312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 669/1333] 10.222.0.57 () {58 vars in 1061 bytes} [Thu May 26 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:04.663272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482be6de-4b11-41a7-a54c-40b69bb7743a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:04.664812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-482be6de-4b11-41a7-a54c-40b69bb7743a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:04.777404 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482be6de-4b11-41a7-a54c-40b69bb7743a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:04.872113 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec7a4b-7554-41de-a80e-2269b90d13d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:04.873337 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec7a4b-7554-41de-a80e-2269b90d13d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:04.873623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec7a4b-7554-41de-a80e-2269b90d13d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:04.873904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec7a4b-7554-41de-a80e-2269b90d13d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:04.875624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2dec7a4b-7554-41de-a80e-2269b90d13d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:04.905257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:04.991174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec7a4b-7554-41de-a80e-2269b90d13d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:04.992623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec7a4b-7554-41de-a80e-2269b90d13d1 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:05.011808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 665/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:07.491527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8914094-0629-403e-a7ae-f5a892ad3e14 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:07.492069 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8914094-0629-403e-a7ae-f5a892ad3e14 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:07.492174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8914094-0629-403e-a7ae-f5a892ad3e14 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:07.493481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 670/1335] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:07.499255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95316f-5292-41ef-9267-facffe7420e7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:07.499648 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95316f-5292-41ef-9267-facffe7420e7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:07.499814 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95316f-5292-41ef-9267-facffe7420e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:07.564189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9e95316f-5292-41ef-9267-facffe7420e7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:07.565382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9e95316f-5292-41ef-9267-facffe7420e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:07.637380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:07.659935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 666/1336] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:07.699985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8914094-0629-403e-a7ae-f5a892ad3e14 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:07.701539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f8914094-0629-403e-a7ae-f5a892ad3e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:07.803460 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8914094-0629-403e-a7ae-f5a892ad3e14 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:07.833470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:07.877221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfacd63c-4e71-492f-87f6-93aeb62ae05c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:07.877875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacd63c-4e71-492f-87f6-93aeb62ae05c service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:07.878158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacd63c-4e71-492f-87f6-93aeb62ae05c service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:07.878459 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacd63c-4e71-492f-87f6-93aeb62ae05c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:07.880115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dfacd63c-4e71-492f-87f6-93aeb62ae05c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:07.985701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacd63c-4e71-492f-87f6-93aeb62ae05c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:07.987104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfacd63c-4e71-492f-87f6-93aeb62ae05c service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:08.005066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 671/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:10.803488 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c46456-16eb-473d-ab64-1dd1a49f24ac None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:10.803991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c46456-16eb-473d-ab64-1dd1a49f24ac None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:10.804100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c46456-16eb-473d-ab64-1dd1a49f24ac None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:10.805409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 667/1338] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:10.810711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53498-50ac-4b8b-a2f4-705c4878179a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:10.811171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53498-50ac-4b8b-a2f4-705c4878179a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:10.811269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53498-50ac-4b8b-a2f4-705c4878179a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:10.874926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-82a53498-50ac-4b8b-a2f4-705c4878179a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:10.876110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-82a53498-50ac-4b8b-a2f4-705c4878179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:10.884377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:10.970662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 672/1339] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:11.006377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c46456-16eb-473d-ab64-1dd1a49f24ac None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:11.007941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-20c46456-16eb-473d-ab64-1dd1a49f24ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:11.110483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c46456-16eb-473d-ab64-1dd1a49f24ac None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:11.182030 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99fad4-dc52-47cc-863f-585dcee73c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:11.182702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99fad4-dc52-47cc-863f-585dcee73c4b service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:11.182992 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99fad4-dc52-47cc-863f-585dcee73c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:11.183273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99fad4-dc52-47cc-863f-585dcee73c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:11.185415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bb99fad4-dc52-47cc-863f-585dcee73c4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:11.220542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:11.292869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99fad4-dc52-47cc-863f-585dcee73c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:11.294861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99fad4-dc52-47cc-863f-585dcee73c4b service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:11.314404 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 668/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:15.577884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52665c2-e82b-4e63-b1d6-1e1d0007e9b4 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:15.578409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52665c2-e82b-4e63-b1d6-1e1d0007e9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:15.578651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52665c2-e82b-4e63-b1d6-1e1d0007e9b4 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:15.579968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 673/1341] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:15.585524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0a37a-3387-4b50-9770-85640cd2b241 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:15.586382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0a37a-3387-4b50-9770-85640cd2b241 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:15.586754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0a37a-3387-4b50-9770-85640cd2b241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:15.656338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fce0a37a-3387-4b50-9770-85640cd2b241 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:15.657520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fce0a37a-3387-4b50-9770-85640cd2b241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:15.749188 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:15.766475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 669/1342] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:15.802012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52665c2-e82b-4e63-b1d6-1e1d0007e9b4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:15.804384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e52665c2-e82b-4e63-b1d6-1e1d0007e9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:15.903815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52665c2-e82b-4e63-b1d6-1e1d0007e9b4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:15.907816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde0458-6af0-469b-867d-17607b4d7937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:15.908472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde0458-6af0-469b-867d-17607b4d7937 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:15.908756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde0458-6af0-469b-867d-17607b4d7937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:15.909032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde0458-6af0-469b-867d-17607b4d7937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:15.918417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde0458-6af0-469b-867d-17607b4d7937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:15.920445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde0458-6af0-469b-867d-17607b4d7937 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:15.929836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 674/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:18.741608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207dae1-0a03-4cb7-8c67-04e1ddbdee89 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:18.742297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207dae1-0a03-4cb7-8c67-04e1ddbdee89 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:18.742297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207dae1-0a03-4cb7-8c67-04e1ddbdee89 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:18.743464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 670/1344] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:18.749365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63943f-6bd3-4328-b2be-d9beb2da5191 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:18.749824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63943f-6bd3-4328-b2be-d9beb2da5191 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:18.749932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63943f-6bd3-4328-b2be-d9beb2da5191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:18.777945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9e63943f-6bd3-4328-b2be-d9beb2da5191 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:18.779311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e63943f-6bd3-4328-b2be-d9beb2da5191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:18.811136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:18.862315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 675/1345] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:18.899667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207dae1-0a03-4cb7-8c67-04e1ddbdee89 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:18.910940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207dae1-0a03-4cb7-8c67-04e1ddbdee89 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:18.914566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d3c61-53d9-409a-a5e4-f3c7b57d55a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:18.915281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d3c61-53d9-409a-a5e4-f3c7b57d55a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:18.915631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d3c61-53d9-409a-a5e4-f3c7b57d55a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:18.915975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d3c61-53d9-409a-a5e4-f3c7b57d55a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:18.924989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d3c61-53d9-409a-a5e4-f3c7b57d55a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:18.926558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d3c61-53d9-409a-a5e4-f3c7b57d55a7 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:18.936403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 671/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:23.056549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024698be-c3f9-41d1-8278-c93172049752 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:23.056983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024698be-c3f9-41d1-8278-c93172049752 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:23.057186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024698be-c3f9-41d1-8278-c93172049752 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:23.058430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 676/1347] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:56:23.063903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9fe1c-d773-4820-98de-ea89eeb96aed None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:23.064367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9fe1c-d773-4820-98de-ea89eeb96aed None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:23.064524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9fe1c-d773-4820-98de-ea89eeb96aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:23.082554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a0f9fe1c-d773-4820-98de-ea89eeb96aed None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:23.083838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a0f9fe1c-d773-4820-98de-ea89eeb96aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:23.092542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 672/1348] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:23.129744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024698be-c3f9-41d1-8278-c93172049752 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:23.140826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024698be-c3f9-41d1-8278-c93172049752 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:23.144319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c9ee3-f4e9-4062-93a6-5c1ac2a30781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:23.144955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9ee3-f4e9-4062-93a6-5c1ac2a30781 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:23.145248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9ee3-f4e9-4062-93a6-5c1ac2a30781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:23.145531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c9ee3-f4e9-4062-93a6-5c1ac2a30781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:23.155033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9ee3-f4e9-4062-93a6-5c1ac2a30781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:23.156472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c9ee3-f4e9-4062-93a6-5c1ac2a30781 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:23.165949 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 677/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:28.374859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565dd92-9da3-44e0-ae5f-48d20f7a6319 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:28.375359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565dd92-9da3-44e0-ae5f-48d20f7a6319 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:28.375441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565dd92-9da3-44e0-ae5f-48d20f7a6319 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:28.377206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 673/1350] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 20:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:28.382762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e21ce-f07b-4ee6-b8b2-1ce17daad221 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:28.383167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e21ce-f07b-4ee6-b8b2-1ce17daad221 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:28.383383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e21ce-f07b-4ee6-b8b2-1ce17daad221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:28.461272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f07e21ce-f07b-4ee6-b8b2-1ce17daad221 None None] MFA Rules not processed for user `4edb53dc871047ce89ebb3b90143a6de`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:28.462572 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f07e21ce-f07b-4ee6-b8b2-1ce17daad221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:28.570215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 678/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 20:56:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:32.865641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dec80-b688-44c9-8588-ce5111c1002b None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:32.866170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dec80-b688-44c9-8588-ce5111c1002b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:32.866424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351dec80-b688-44c9-8588-ce5111c1002b None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:32.868103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 674/1352] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 20:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:32.873095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832beb9-edb3-4ef4-a1bd-320cb309d002 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:32.873531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832beb9-edb3-4ef4-a1bd-320cb309d002 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:32.873663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832beb9-edb3-4ef4-a1bd-320cb309d002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:32.894045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1832beb9-edb3-4ef4-a1bd-320cb309d002 None None] MFA Rules not processed for user `4edb53dc871047ce89ebb3b90143a6de`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:32.895602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1832beb9-edb3-4ef4-a1bd-320cb309d002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:32.907204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 679/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 20:56:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:33.077506 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=13 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:34.592111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5bb83-6c9f-4fc4-833e-f9ffd43f473f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:34.592869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5bb83-6c9f-4fc4-833e-f9ffd43f473f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:34.592869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee5bb83-6c9f-4fc4-833e-f9ffd43f473f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:34.594085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 675/1354] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 20:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:34.599042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05327d-39cd-4264-8e6f-9afd17c37cf4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:34.599498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05327d-39cd-4264-8e6f-9afd17c37cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:34.599591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05327d-39cd-4264-8e6f-9afd17c37cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:34.617523 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-de05327d-39cd-4264-8e6f-9afd17c37cf4 None None] MFA Rules not processed for user `4edb53dc871047ce89ebb3b90143a6de`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:34.618734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-de05327d-39cd-4264-8e6f-9afd17c37cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:34.627243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 680/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 20:56:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:38.399091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010de77-69d4-4d6b-8b3b-38df6044fd08 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:38.399624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010de77-69d4-4d6b-8b3b-38df6044fd08 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:38.399863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010de77-69d4-4d6b-8b3b-38df6044fd08 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:38.401510 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 676/1356] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:38.406525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca22b2-eb1f-4b71-9aed-9016ad47803f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:38.406910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca22b2-eb1f-4b71-9aed-9016ad47803f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:38.407091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca22b2-eb1f-4b71-9aed-9016ad47803f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:38.425520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bfca22b2-eb1f-4b71-9aed-9016ad47803f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:38.426961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bfca22b2-eb1f-4b71-9aed-9016ad47803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:38.435517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 681/1357] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:38.487956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85187a23-3137-45c2-9713-c61af6f30a39 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:38.488247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85187a23-3137-45c2-9713-c61af6f30a39 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:38.488521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85187a23-3137-45c2-9713-c61af6f30a39 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:38.490056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 677/1358] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 26 20:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:38.498140 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f7585-de66-43f5-9626-a6f24d856659 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:38.498438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f7585-de66-43f5-9626-a6f24d856659 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:38.498748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f7585-de66-43f5-9626-a6f24d856659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:38.569781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:38.579104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-260f7585-de66-43f5-9626-a6f24d856659 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:38.580313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-260f7585-de66-43f5-9626-a6f24d856659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:38.667097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 682/1359] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 26 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:38.674657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85187a23-3137-45c2-9713-c61af6f30a39 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:38.689709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85187a23-3137-45c2-9713-c61af6f30a39 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:38.695476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c596bb-b363-454e-aeb9-ed8ffa0b65bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:38.696114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c596bb-b363-454e-aeb9-ed8ffa0b65bd service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:38.696400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c596bb-b363-454e-aeb9-ed8ffa0b65bd service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:38.696861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c596bb-b363-454e-aeb9-ed8ffa0b65bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:38.706001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c596bb-b363-454e-aeb9-ed8ffa0b65bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:38.707771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c596bb-b363-454e-aeb9-ed8ffa0b65bd service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:38.717541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 678/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:41.155111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ee41d-4b92-49bc-9569-5563a2f7285c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:41.155644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ee41d-4b92-49bc-9569-5563a2f7285c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:41.155644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ee41d-4b92-49bc-9569-5563a2f7285c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:41.157095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 683/1361] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:41.162150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fd8e0-6640-41c3-bffc-97abe61a8d50 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:41.162568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fd8e0-6640-41c3-bffc-97abe61a8d50 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:41.162747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fd8e0-6640-41c3-bffc-97abe61a8d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:41.181113 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8d5fd8e0-6640-41c3-bffc-97abe61a8d50 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:41.182560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8d5fd8e0-6640-41c3-bffc-97abe61a8d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:41.190613 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 679/1362] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:41.235149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ee41d-4b92-49bc-9569-5563a2f7285c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:41.249923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ee41d-4b92-49bc-9569-5563a2f7285c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:41.255766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9127bd5-837f-494c-a074-34ed169ab5f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:41.256416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9127bd5-837f-494c-a074-34ed169ab5f3 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:41.257204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9127bd5-837f-494c-a074-34ed169ab5f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:41.257204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9127bd5-837f-494c-a074-34ed169ab5f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:41.265617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9127bd5-837f-494c-a074-34ed169ab5f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:41.267251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9127bd5-837f-494c-a074-34ed169ab5f3 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:41.277303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 684/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:43.788935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812a661-bba7-4985-841c-8bec8c9ae81e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:43.789820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812a661-bba7-4985-841c-8bec8c9ae81e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:43.790109 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812a661-bba7-4985-841c-8bec8c9ae81e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:43.791548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 680/1364] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:43.797471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e9651-5c42-46d0-a4b8-7e08020571c0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:43.797933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e9651-5c42-46d0-a4b8-7e08020571c0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:43.798029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e9651-5c42-46d0-a4b8-7e08020571c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:43.820942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-af9e9651-5c42-46d0-a4b8-7e08020571c0 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:43.822157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-af9e9651-5c42-46d0-a4b8-7e08020571c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:43.830363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 685/1365] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:43.876856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd4aff-8843-46dd-9bb8-77b463469f46 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:43.878328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd4aff-8843-46dd-9bb8-77b463469f46 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:43.878759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd4aff-8843-46dd-9bb8-77b463469f46 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:43.880128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 681/1366] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 26 20:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:43.888331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbe028-24fc-4223-ba89-0206795b5740 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:43.888628 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbe028-24fc-4223-ba89-0206795b5740 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:43.888909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbe028-24fc-4223-ba89-0206795b5740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:43.917172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-42cbe028-24fc-4223-ba89-0206795b5740 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:43.918657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-42cbe028-24fc-4223-ba89-0206795b5740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:43.929013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 686/1367] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 26 20:56:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:43.937404 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd4aff-8843-46dd-9bb8-77b463469f46 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:43.951766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd4aff-8843-46dd-9bb8-77b463469f46 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:43.957148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c535dbb-96f6-43a3-b9bc-da1259625e2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:43.957783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c535dbb-96f6-43a3-b9bc-da1259625e2d service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:43.958066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c535dbb-96f6-43a3-b9bc-da1259625e2d service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:43.958345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c535dbb-96f6-43a3-b9bc-da1259625e2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:43.967018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c535dbb-96f6-43a3-b9bc-da1259625e2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:43.968606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c535dbb-96f6-43a3-b9bc-da1259625e2d service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:43.978047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 682/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:46.249084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aec50-25e4-401f-97b0-07ecd4a005b3 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:46.249654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aec50-25e4-401f-97b0-07ecd4a005b3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:46.249654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796aec50-25e4-401f-97b0-07ecd4a005b3 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:46.251039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 687/1369] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:46.256976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cffc98-4674-4934-a323-e1dce4b7f1df None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:46.257424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cffc98-4674-4934-a323-e1dce4b7f1df None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:46.257740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cffc98-4674-4934-a323-e1dce4b7f1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:46.276607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-90cffc98-4674-4934-a323-e1dce4b7f1df None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:46.277817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-90cffc98-4674-4934-a323-e1dce4b7f1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:46.286875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 683/1370] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:46.331119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796aec50-25e4-401f-97b0-07ecd4a005b3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:46.346591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796aec50-25e4-401f-97b0-07ecd4a005b3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:46.352181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5cfc2-1ade-4640-a380-69bfbc0fae51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:46.352946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5cfc2-1ade-4640-a380-69bfbc0fae51 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:46.353354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5cfc2-1ade-4640-a380-69bfbc0fae51 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:46.353741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5cfc2-1ade-4640-a380-69bfbc0fae51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:46.363344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5cfc2-1ade-4640-a380-69bfbc0fae51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:46.365070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5cfc2-1ade-4640-a380-69bfbc0fae51 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:46.375144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 688/1371] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:46.647506 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc28fa0-3e1d-4808-afa6-95c07e0c243d None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:46.647861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc28fa0-3e1d-4808-afa6-95c07e0c243d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:46.648076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc28fa0-3e1d-4808-afa6-95c07e0c243d None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:46.649564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 684/1372] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:46.656066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237081a6-1718-4609-8976-fa7e1e212085 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:46.656362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237081a6-1718-4609-8976-fa7e1e212085 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:46.656828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237081a6-1718-4609-8976-fa7e1e212085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:46.675419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-237081a6-1718-4609-8976-fa7e1e212085 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:46.676643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-237081a6-1718-4609-8976-fa7e1e212085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:46.684866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 689/1373] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:46.997548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538941dc-00c4-49f5-8238-87f1baac1a79 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:46.997548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538941dc-00c4-49f5-8238-87f1baac1a79 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:46.997548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538941dc-00c4-49f5-8238-87f1baac1a79 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:46.997548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 685/1374] 10.222.0.57 () {54 vars in 963 bytes} [Thu May 26 20:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:47.006019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716f348-dbd8-4314-816a-1df28adf6a03 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:47.006585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716f348-dbd8-4314-816a-1df28adf6a03 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:47.006883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716f348-dbd8-4314-816a-1df28adf6a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:47.056790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c716f348-dbd8-4314-816a-1df28adf6a03 None None] MFA Rules not processed for user `afe351334485400a8c11f17492c9ff5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:47.058213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c716f348-dbd8-4314-816a-1df28adf6a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:47.121408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 690/1375] 10.222.0.57 () {58 vars in 1060 bytes} [Thu May 26 20:56:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:47.129601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538941dc-00c4-49f5-8238-87f1baac1a79 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:47.141759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538941dc-00c4-49f5-8238-87f1baac1a79 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:47.145932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaab2ab-0c69-47bc-a110-2e8b37d1fdc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:47.146869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaab2ab-0c69-47bc-a110-2e8b37d1fdc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:47.147378 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaab2ab-0c69-47bc-a110-2e8b37d1fdc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:47.147903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaab2ab-0c69-47bc-a110-2e8b37d1fdc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:47.158044 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaab2ab-0c69-47bc-a110-2e8b37d1fdc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:47.159685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaab2ab-0c69-47bc-a110-2e8b37d1fdc8 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:47.169718 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 686/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:48.622796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f4568-4a20-44d5-871e-ea0a453c8c82 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:48.623392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f4568-4a20-44d5-871e-ea0a453c8c82 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:48.623392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f4568-4a20-44d5-871e-ea0a453c8c82 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:48.624745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 691/1377] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:48.630076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac2080-d5fb-4702-adbc-ce6e7fcfe924 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:48.630821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac2080-d5fb-4702-adbc-ce6e7fcfe924 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:48.631040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac2080-d5fb-4702-adbc-ce6e7fcfe924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:48.648358 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d7ac2080-d5fb-4702-adbc-ce6e7fcfe924 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:48.649550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d7ac2080-d5fb-4702-adbc-ce6e7fcfe924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:48.659273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 687/1378] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:48.704846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f4568-4a20-44d5-871e-ea0a453c8c82 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:48.720749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f4568-4a20-44d5-871e-ea0a453c8c82 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:48.726432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8da10d-f71d-432e-80ce-4811f94affbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:48.727234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8da10d-f71d-432e-80ce-4811f94affbb service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:48.727526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8da10d-f71d-432e-80ce-4811f94affbb service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:48.727806 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8da10d-f71d-432e-80ce-4811f94affbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:48.737570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8da10d-f71d-432e-80ce-4811f94affbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:48.739227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8da10d-f71d-432e-80ce-4811f94affbb service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:48.743680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:56:48.751652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 692/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:51.025697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71505e-3fb0-4356-87ca-0c50062159a5 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:51.026239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71505e-3fb0-4356-87ca-0c50062159a5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:51.026239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71505e-3fb0-4356-87ca-0c50062159a5 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:51.027560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 688/1380] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:51.033047 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890f6d6-dfea-4ab8-b901-188a3787498a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:51.033501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890f6d6-dfea-4ab8-b901-188a3787498a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:51.033601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890f6d6-dfea-4ab8-b901-188a3787498a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:51.052099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f890f6d6-dfea-4ab8-b901-188a3787498a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:51.053289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f890f6d6-dfea-4ab8-b901-188a3787498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:51.066796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 693/1381] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:51.109107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71505e-3fb0-4356-87ca-0c50062159a5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:51.124699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71505e-3fb0-4356-87ca-0c50062159a5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:51.130262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d881a-f533-449f-8b42-87f54cb92507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:51.131291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d881a-f533-449f-8b42-87f54cb92507 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:51.131641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d881a-f533-449f-8b42-87f54cb92507 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:51.132779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d881a-f533-449f-8b42-87f54cb92507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:51.142341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d881a-f533-449f-8b42-87f54cb92507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:51.144038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d881a-f533-449f-8b42-87f54cb92507 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:51.154539 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 689/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:53.371783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7c275-ff05-4e27-ab60-b4dbb6d0b0b9 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:53.372304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7c275-ff05-4e27-ab60-b4dbb6d0b0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:53.372404 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e7c275-ff05-4e27-ab60-b4dbb6d0b0b9 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:53.373693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 694/1383] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:53.379169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c9c8-7c49-41bc-bc4c-1b36cabbc391 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:53.379590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c9c8-7c49-41bc-bc4c-1b36cabbc391 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:53.379727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb1c9c8-7c49-41bc-bc4c-1b36cabbc391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:53.399638 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-eeb1c9c8-7c49-41bc-bc4c-1b36cabbc391 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:53.400898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eeb1c9c8-7c49-41bc-bc4c-1b36cabbc391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:53.410104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 690/1384] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:53.452018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7c275-ff05-4e27-ab60-b4dbb6d0b0b9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:53.468873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e7c275-ff05-4e27-ab60-b4dbb6d0b0b9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:53.474889 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc6c8c-556d-4e23-9685-6452417e0d76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:53.475544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc6c8c-556d-4e23-9685-6452417e0d76 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:53.475835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc6c8c-556d-4e23-9685-6452417e0d76 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:53.476120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc6c8c-556d-4e23-9685-6452417e0d76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:53.485066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc6c8c-556d-4e23-9685-6452417e0d76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:53.486635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bc6c8c-556d-4e23-9685-6452417e0d76 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:53.496587 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 695/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:55.840512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea804-4aca-4aee-93f0-2acfa3555cd3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:55.841026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea804-4aca-4aee-93f0-2acfa3555cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:55.841125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cea804-4aca-4aee-93f0-2acfa3555cd3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:55.842457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 691/1386] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:56:55.847644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e1c17-6202-4db0-b4ed-17715dfc86c8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:55.848062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e1c17-6202-4db0-b4ed-17715dfc86c8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:55.848216 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e1c17-6202-4db0-b4ed-17715dfc86c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:55.866134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ec9e1c17-6202-4db0-b4ed-17715dfc86c8 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:55.867378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ec9e1c17-6202-4db0-b4ed-17715dfc86c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:55.876159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 696/1387] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:55.920760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cea804-4aca-4aee-93f0-2acfa3555cd3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:55.934961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cea804-4aca-4aee-93f0-2acfa3555cd3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:55.940811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6237b1-0163-4f15-b09f-f3973fa66c4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:55.941472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6237b1-0163-4f15-b09f-f3973fa66c4c service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:55.941756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6237b1-0163-4f15-b09f-f3973fa66c4c service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:55.942032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6237b1-0163-4f15-b09f-f3973fa66c4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:55.950366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6237b1-0163-4f15-b09f-f3973fa66c4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:55.951990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6237b1-0163-4f15-b09f-f3973fa66c4c service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:55.961527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 692/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:56:58.260811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18380d69-0278-4299-b8e2-7249994ca9b7 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:58.261451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18380d69-0278-4299-b8e2-7249994ca9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:58.261451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18380d69-0278-4299-b8e2-7249994ca9b7 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:58.263353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 697/1389] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:56:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 20:56:58.268731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ced36-c97e-451d-87af-e0880a663195 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:58.269209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ced36-c97e-451d-87af-e0880a663195 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:58.269360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ced36-c97e-451d-87af-e0880a663195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:58.287357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-696ced36-c97e-451d-87af-e0880a663195 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:56:58.288556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-696ced36-c97e-451d-87af-e0880a663195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:56:58.299547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 693/1390] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:56:58.346392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18380d69-0278-4299-b8e2-7249994ca9b7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:56:58.364627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18380d69-0278-4299-b8e2-7249994ca9b7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:56:58.371630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a5f72-ed07-4444-ad17-ef6303fc53bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:56:58.373242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a5f72-ed07-4444-ad17-ef6303fc53bc service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:56:58.373621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a5f72-ed07-4444-ad17-ef6303fc53bc service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:56:58.373979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a5f72-ed07-4444-ad17-ef6303fc53bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:56:58.385808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a5f72-ed07-4444-ad17-ef6303fc53bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:56:58.387883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a5f72-ed07-4444-ad17-ef6303fc53bc service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:56:58.400253 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 698/1391] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:56:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:00.726100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687e3d8-5bc9-4abd-a6ae-5ff93f39fd9c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:00.726694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687e3d8-5bc9-4abd-a6ae-5ff93f39fd9c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:00.726951 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687e3d8-5bc9-4abd-a6ae-5ff93f39fd9c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:00.728700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 694/1392] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:57:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:00.734113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86db0a-25a4-453f-810e-31719764d068 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:00.734523 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86db0a-25a4-453f-810e-31719764d068 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:00.734744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86db0a-25a4-453f-810e-31719764d068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:00.758579 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-be86db0a-25a4-453f-810e-31719764d068 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:00.759845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-be86db0a-25a4-453f-810e-31719764d068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:00.770415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 699/1393] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:00.825566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687e3d8-5bc9-4abd-a6ae-5ff93f39fd9c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:00.841415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687e3d8-5bc9-4abd-a6ae-5ff93f39fd9c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:00.846981 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d489014e-1f7c-45cf-84bc-92d003c6bf73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:00.847649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489014e-1f7c-45cf-84bc-92d003c6bf73 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:00.847934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489014e-1f7c-45cf-84bc-92d003c6bf73 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:00.848218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d489014e-1f7c-45cf-84bc-92d003c6bf73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:00.856971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489014e-1f7c-45cf-84bc-92d003c6bf73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:00.858601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d489014e-1f7c-45cf-84bc-92d003c6bf73 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:00.868223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 695/1394] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:03.023976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e6c09-90c8-43af-90c8-33b97544038c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.024531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e6c09-90c8-43af-90c8-33b97544038c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.024531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e6c09-90c8-43af-90c8-33b97544038c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:03.026055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 700/1395] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:03.031123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfa2f7-0f2b-4c21-9f55-e61891aae590 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.031566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfa2f7-0f2b-4c21-9f55-e61891aae590 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.031675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfa2f7-0f2b-4c21-9f55-e61891aae590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:03.049561 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b1bfa2f7-0f2b-4c21-9f55-e61891aae590 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:03.050770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b1bfa2f7-0f2b-4c21-9f55-e61891aae590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:03.058369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 696/1396] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:03.107865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871804b-3158-4699-baff-2c5e4c6da363 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.108161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871804b-3158-4699-baff-2c5e4c6da363 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.108441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871804b-3158-4699-baff-2c5e4c6da363 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:03.109838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 701/1397] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:03.114624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9c088-18d2-476b-b7b0-aa249698b21c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.114813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca461a-9e81-49c1-8fb3-e6d57503ac97 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.114938 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9c088-18d2-476b-b7b0-aa249698b21c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.115209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9c088-18d2-476b-b7b0-aa249698b21c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:03.115311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca461a-9e81-49c1-8fb3-e6d57503ac97 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.115611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca461a-9e81-49c1-8fb3-e6d57503ac97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:03.116541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 697/1398] 10.222.0.57 () {54 vars in 962 bytes} [Thu May 26 20:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:03.121347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b5c7-570f-4e1c-af55-b7fdafb3394c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.122253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b5c7-570f-4e1c-af55-b7fdafb3394c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.122588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b5c7-570f-4e1c-af55-b7fdafb3394c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:03.128060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=55 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:57:03.135163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-11ca461a-9e81-49c1-8fb3-e6d57503ac97 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:03.136385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-11ca461a-9e81-49c1-8fb3-e6d57503ac97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:03.144338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 702/1399] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:57:03.172975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-34d8b5c7-570f-4e1c-af55-b7fdafb3394c None None] MFA Rules not processed for user `2c3bd8dd462846bf9423f8528aa5c489`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:03.174172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-34d8b5c7-570f-4e1c-af55-b7fdafb3394c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:03.188472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca461a-9e81-49c1-8fb3-e6d57503ac97 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:03.200315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca461a-9e81-49c1-8fb3-e6d57503ac97 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:03.205692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613fb99-f887-4b1d-9f48-163ece8e54be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:03.206356 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613fb99-f887-4b1d-9f48-163ece8e54be service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.206674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613fb99-f887-4b1d-9f48-163ece8e54be service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.206956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613fb99-f887-4b1d-9f48-163ece8e54be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:03.215370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613fb99-f887-4b1d-9f48-163ece8e54be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:03.216944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613fb99-f887-4b1d-9f48-163ece8e54be service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:03.226440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 703/1400] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 20:57:03.233493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 698/1401] 10.222.0.57 () {58 vars in 1059 bytes} [Thu May 26 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:57:03.241168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613fb99-f887-4b1d-9f48-163ece8e54be service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:03.250499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613fb99-f887-4b1d-9f48-163ece8e54be service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:03.254268 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6f3cf-8ad9-48ac-a348-ac0bb86f64bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3bd8dd462846bf9423f8528aa5c489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:03.254950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6f3cf-8ad9-48ac-a348-ac0bb86f64bd service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.255238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6f3cf-8ad9-48ac-a348-ac0bb86f64bd service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.255782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6f3cf-8ad9-48ac-a348-ac0bb86f64bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:03.265408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6f3cf-8ad9-48ac-a348-ac0bb86f64bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:03.266892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6f3cf-8ad9-48ac-a348-ac0bb86f64bd service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:03.276403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 704/1402] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:03.814061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51bbb7-86a3-4b8e-9268-57f30fcca695 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.814368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51bbb7-86a3-4b8e-9268-57f30fcca695 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.814670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51bbb7-86a3-4b8e-9268-57f30fcca695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:03.863997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fe51bbb7-86a3-4b8e-9268-57f30fcca695 None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:03.865220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fe51bbb7-86a3-4b8e-9268-57f30fcca695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:03.929679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 699/1403] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:03.973138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dea1d-6ee6-4f35-97b4-8fe3e01ce438 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.973563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dea1d-6ee6-4f35-97b4-8fe3e01ce438 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.973929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dea1d-6ee6-4f35-97b4-8fe3e01ce438 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:03.975447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 705/1404] 10.222.0.57 () {54 vars in 942 bytes} [Thu May 26 20:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:03.982570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8f8a-2345-4aea-aed4-0b1418e42c5a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:03.982864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8f8a-2345-4aea-aed4-0b1418e42c5a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:03.983146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b8f8a-2345-4aea-aed4-0b1418e42c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:04.055018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-803b8f8a-2345-4aea-aed4-0b1418e42c5a None None] MFA Rules not processed for user `1285406084604bd19287de9e1beeeaa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:04.061083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-803b8f8a-2345-4aea-aed4-0b1418e42c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:04.087909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:57:04.144422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 700/1405] 10.222.0.57 () {58 vars in 1039 bytes} [Thu May 26 20:57:03 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:04.154237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dea1d-6ee6-4f35-97b4-8fe3e01ce438 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:04.169221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dea1d-6ee6-4f35-97b4-8fe3e01ce438 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:04.174213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebac2c-d243-4774-833e-28f30c4b29cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:04.174892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebac2c-d243-4774-833e-28f30c4b29cb service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:04.175183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebac2c-d243-4774-833e-28f30c4b29cb service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:04.175465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebac2c-d243-4774-833e-28f30c4b29cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:04.183437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebac2c-d243-4774-833e-28f30c4b29cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:04.185557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adebac2c-d243-4774-833e-28f30c4b29cb service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:04.193646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 706/1406] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 20:57:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 20:57:05.467777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2803f31-577e-4455-84ad-35d5921728ea None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:05.468377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2803f31-577e-4455-84ad-35d5921728ea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:05.468377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2803f31-577e-4455-84ad-35d5921728ea None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:05.469698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 701/1407] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:05.475095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e84739-6881-4bda-b88c-2531fcac2f5a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:05.475882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e84739-6881-4bda-b88c-2531fcac2f5a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:05.476314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e84739-6881-4bda-b88c-2531fcac2f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:05.500756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-04e84739-6881-4bda-b88c-2531fcac2f5a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:05.501993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-04e84739-6881-4bda-b88c-2531fcac2f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:05.510861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 707/1408] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:05.555032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2803f31-577e-4455-84ad-35d5921728ea None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:05.570776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2803f31-577e-4455-84ad-35d5921728ea None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:05.576302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e502158-7066-4302-a601-2966dfce7615 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:05.576947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e502158-7066-4302-a601-2966dfce7615 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:05.577232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e502158-7066-4302-a601-2966dfce7615 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:05.577507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e502158-7066-4302-a601-2966dfce7615 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:05.585800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e502158-7066-4302-a601-2966dfce7615 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:05.589469 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e502158-7066-4302-a601-2966dfce7615 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:05.600148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 702/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:15.492387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=46 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:57:15.595117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=27 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:57:27.633008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c57c3-98ea-47c9-85a8-d50fc66b2ac0 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:27.633649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c57c3-98ea-47c9-85a8-d50fc66b2ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:27.633649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c57c3-98ea-47c9-85a8-d50fc66b2ac0 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:27.634987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 708/1410] 10.222.0.57 () {54 vars in 918 bytes} [Thu May 26 20:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:27.640031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cb75b-c145-4bb2-9132-b5444e708212 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:27.640510 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cb75b-c145-4bb2-9132-b5444e708212 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:27.640626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133cb75b-c145-4bb2-9132-b5444e708212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:27.658648 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-133cb75b-c145-4bb2-9132-b5444e708212 None None] MFA Rules not processed for user `4edb53dc871047ce89ebb3b90143a6de`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:27.659824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-133cb75b-c145-4bb2-9132-b5444e708212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:27.668754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 703/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Thu May 26 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:27.682458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9651be-3b8c-4d34-9065-bf7b6bd361f9 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:27.682772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9651be-3b8c-4d34-9065-bf7b6bd361f9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:27.683058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9651be-3b8c-4d34-9065-bf7b6bd361f9 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:27.684424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 709/1412] 10.222.0.57 () {54 vars in 918 bytes} [Thu May 26 20:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:27.689031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc2755-c659-4001-8ab0-17a872211ebb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:27.689324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc2755-c659-4001-8ab0-17a872211ebb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:27.689611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc2755-c659-4001-8ab0-17a872211ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:27.707681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e7bc2755-c659-4001-8ab0-17a872211ebb None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:27.708897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e7bc2755-c659-4001-8ab0-17a872211ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:27.719126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 704/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Thu May 26 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:30.449130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306729bc-d420-4a40-a201-01779177abb5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:30.449669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306729bc-d420-4a40-a201-01779177abb5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:30.449669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306729bc-d420-4a40-a201-01779177abb5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:30.451597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:57:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:30.457157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe0bc-1b0c-4084-b262-c78b9bd2372c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:30.457642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe0bc-1b0c-4084-b262-c78b9bd2372c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:30.457753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fe0bc-1b0c-4084-b262-c78b9bd2372c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:30.476201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e88fe0bc-1b0c-4084-b262-c78b9bd2372c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:30.477397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e88fe0bc-1b0c-4084-b262-c78b9bd2372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:30.485156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:57:30.528225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306729bc-d420-4a40-a201-01779177abb5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:30.542964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306729bc-d420-4a40-a201-01779177abb5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:30.548285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8fcc73-7b6b-4c15-a9ae-50ff41dd2835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:30.548921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fcc73-7b6b-4c15-a9ae-50ff41dd2835 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:30.549215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fcc73-7b6b-4c15-a9ae-50ff41dd2835 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:30.549499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8fcc73-7b6b-4c15-a9ae-50ff41dd2835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:30.557438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fcc73-7b6b-4c15-a9ae-50ff41dd2835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:30.559619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8fcc73-7b6b-4c15-a9ae-50ff41dd2835 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:30.568923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 711/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:31.538067 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178d0bb-0d94-427d-af4f-524125b09ba4 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:31.538642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178d0bb-0d94-427d-af4f-524125b09ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:31.538642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178d0bb-0d94-427d-af4f-524125b09ba4 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:31.540017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:31.545520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998563a-786e-436e-99da-f1ba4a757479 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:31.545946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998563a-786e-436e-99da-f1ba4a757479 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:31.546091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998563a-786e-436e-99da-f1ba4a757479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:31.563764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7998563a-786e-436e-99da-f1ba4a757479 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:31.564965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7998563a-786e-436e-99da-f1ba4a757479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:31.573141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:31.726559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178d0bb-0d94-427d-af4f-524125b09ba4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:31.745993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178d0bb-0d94-427d-af4f-524125b09ba4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:31.752492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579c098-556e-494f-9d0f-52abbd84fe85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:31.753498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579c098-556e-494f-9d0f-52abbd84fe85 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:31.753891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579c098-556e-494f-9d0f-52abbd84fe85 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:31.754172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579c098-556e-494f-9d0f-52abbd84fe85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:31.766207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579c098-556e-494f-9d0f-52abbd84fe85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:31.767872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0579c098-556e-494f-9d0f-52abbd84fe85 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:31.782821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 707/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:32.707204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4902a8-1736-4915-9d8a-45cab93ec265 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:32.707777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4902a8-1736-4915-9d8a-45cab93ec265 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:32.707777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4902a8-1736-4915-9d8a-45cab93ec265 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:32.709162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:32.713958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ab485-9796-4958-b0e4-d881fab7da8a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:32.714383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ab485-9796-4958-b0e4-d881fab7da8a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:32.714605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ab485-9796-4958-b0e4-d881fab7da8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:32.734158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6a4ab485-9796-4958-b0e4-d881fab7da8a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:32.735406 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6a4ab485-9796-4958-b0e4-d881fab7da8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:32.743124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:32.893519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4902a8-1736-4915-9d8a-45cab93ec265 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:32.908109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4902a8-1736-4915-9d8a-45cab93ec265 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:32.913446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d77d4d-9f1d-43c3-a237-4c85011ac7da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:32.914082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d77d4d-9f1d-43c3-a237-4c85011ac7da service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:32.914369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d77d4d-9f1d-43c3-a237-4c85011ac7da service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:32.914676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d77d4d-9f1d-43c3-a237-4c85011ac7da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:32.923159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d77d4d-9f1d-43c3-a237-4c85011ac7da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:32.925058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d77d4d-9f1d-43c3-a237-4c85011ac7da service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:32.934321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 714/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:33.846294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200abf0-9f5b-4e52-9088-6af8155a2980 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:33.846765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200abf0-9f5b-4e52-9088-6af8155a2980 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:33.846880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200abf0-9f5b-4e52-9088-6af8155a2980 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:33.848239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:33.853356 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f8a9-cd9e-41cf-a728-f8f25c917b2d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:33.853821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f8a9-cd9e-41cf-a728-f8f25c917b2d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:33.853916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653f8a9-cd9e-41cf-a728-f8f25c917b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:33.872053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2653f8a9-cd9e-41cf-a728-f8f25c917b2d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:33.873228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2653f8a9-cd9e-41cf-a728-f8f25c917b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:33.881134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:34.028643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200abf0-9f5b-4e52-9088-6af8155a2980 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:34.043345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6200abf0-9f5b-4e52-9088-6af8155a2980 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:34.048680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f896f-0966-457f-81bf-8caa36f4b7c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:34.049307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f896f-0966-457f-81bf-8caa36f4b7c0 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:34.049590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f896f-0966-457f-81bf-8caa36f4b7c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:34.049872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f896f-0966-457f-81bf-8caa36f4b7c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:34.057838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f896f-0966-457f-81bf-8caa36f4b7c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:34.060061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f896f-0966-457f-81bf-8caa36f4b7c0 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:34.069399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 710/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:34.995349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5da680-67cb-4c22-838b-f2e3281d5966 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:34.995891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5da680-67cb-4c22-838b-f2e3281d5966 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:34.995891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5da680-67cb-4c22-838b-f2e3281d5966 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:34.997725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:57:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:35.002635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbe843-d9c9-4519-a342-1fbaf437e194 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:35.003038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbe843-d9c9-4519-a342-1fbaf437e194 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:35.003186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbe843-d9c9-4519-a342-1fbaf437e194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:35.021986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-62fbe843-d9c9-4519-a342-1fbaf437e194 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:35.023211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-62fbe843-d9c9-4519-a342-1fbaf437e194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:35.030967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:35.183366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5da680-67cb-4c22-838b-f2e3281d5966 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:35.197943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5da680-67cb-4c22-838b-f2e3281d5966 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:35.203888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059de8a-efdb-4335-b51e-8e42ea711125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:35.204521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059de8a-efdb-4335-b51e-8e42ea711125 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:35.204806 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059de8a-efdb-4335-b51e-8e42ea711125 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:35.205084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059de8a-efdb-4335-b51e-8e42ea711125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:35.213017 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059de8a-efdb-4335-b51e-8e42ea711125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:35.214635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059de8a-efdb-4335-b51e-8e42ea711125 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:35.223876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 717/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:36.057967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeddc3e-f62d-4710-a7f2-aac5fff9b136 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:36.058547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeddc3e-f62d-4710-a7f2-aac5fff9b136 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:36.059139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeddc3e-f62d-4710-a7f2-aac5fff9b136 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:36.060443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:36.065347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1268afc-00ae-477d-bdf0-6ed7c0a196e6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:36.065745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1268afc-00ae-477d-bdf0-6ed7c0a196e6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:36.065839 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1268afc-00ae-477d-bdf0-6ed7c0a196e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:36.083434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e1268afc-00ae-477d-bdf0-6ed7c0a196e6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:36.084625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e1268afc-00ae-477d-bdf0-6ed7c0a196e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:36.092450 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:36.240551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeddc3e-f62d-4710-a7f2-aac5fff9b136 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:36.255483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeddc3e-f62d-4710-a7f2-aac5fff9b136 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:36.260742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a431d98-4bb4-4fde-bc1c-1e449afcd808 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:36.261369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a431d98-4bb4-4fde-bc1c-1e449afcd808 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:36.261651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a431d98-4bb4-4fde-bc1c-1e449afcd808 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:36.261924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a431d98-4bb4-4fde-bc1c-1e449afcd808 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:36.269832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a431d98-4bb4-4fde-bc1c-1e449afcd808 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:36.271443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a431d98-4bb4-4fde-bc1c-1e449afcd808 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:36.280598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 713/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:37.116631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98346034-5321-42e4-b406-3f98c04baaf4 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:37.117164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98346034-5321-42e4-b406-3f98c04baaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:37.117164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98346034-5321-42e4-b406-3f98c04baaf4 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:37.118491 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:37.123361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d173b-30a1-44cc-858c-7cc5c5f8bcbe None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:37.123763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d173b-30a1-44cc-858c-7cc5c5f8bcbe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:37.123918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d173b-30a1-44cc-858c-7cc5c5f8bcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:37.142244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0a7d173b-30a1-44cc-858c-7cc5c5f8bcbe None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:37.143454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a7d173b-30a1-44cc-858c-7cc5c5f8bcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:37.151367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:37.298526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98346034-5321-42e4-b406-3f98c04baaf4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:37.312995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98346034-5321-42e4-b406-3f98c04baaf4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:37.318244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022d403-6773-4250-a515-e183841b6e91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:37.318915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022d403-6773-4250-a515-e183841b6e91 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:37.319200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022d403-6773-4250-a515-e183841b6e91 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:37.319472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022d403-6773-4250-a515-e183841b6e91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:37.327940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022d403-6773-4250-a515-e183841b6e91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:37.329526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022d403-6773-4250-a515-e183841b6e91 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:37.338637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 720/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:38.172912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46f5f7-48c8-4f99-a065-4c8d0f34157c None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:38.173419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46f5f7-48c8-4f99-a065-4c8d0f34157c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:38.173419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46f5f7-48c8-4f99-a065-4c8d0f34157c None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:38.174777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:38.179562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba93e0-9c30-44a0-9a36-1c4c67a559a6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:38.179997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba93e0-9c30-44a0-9a36-1c4c67a559a6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:38.180154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba93e0-9c30-44a0-9a36-1c4c67a559a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:38.201421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-82ba93e0-9c30-44a0-9a36-1c4c67a559a6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:38.202935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-82ba93e0-9c30-44a0-9a36-1c4c67a559a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:38.211830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:38.360561 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46f5f7-48c8-4f99-a065-4c8d0f34157c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:38.375171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46f5f7-48c8-4f99-a065-4c8d0f34157c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:38.381128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f7913-af45-43a1-8eae-ae1c3fbe50bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:38.381765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f7913-af45-43a1-8eae-ae1c3fbe50bd service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:38.382049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f7913-af45-43a1-8eae-ae1c3fbe50bd service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:38.382324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f7913-af45-43a1-8eae-ae1c3fbe50bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:38.391146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f7913-af45-43a1-8eae-ae1c3fbe50bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:38.392739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f7913-af45-43a1-8eae-ae1c3fbe50bd service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:38.402016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 716/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:39.244280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf2452-5010-46fa-9d32-3cdda508ec07 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:39.244815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf2452-5010-46fa-9d32-3cdda508ec07 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:39.244815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf2452-5010-46fa-9d32-3cdda508ec07 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:39.246157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 20:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:39.251031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bc7e0-69d2-4e31-a6de-be9d2a05812f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:39.251451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bc7e0-69d2-4e31-a6de-be9d2a05812f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:39.251597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bc7e0-69d2-4e31-a6de-be9d2a05812f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:39.269939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-115bc7e0-69d2-4e31-a6de-be9d2a05812f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:39.271194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-115bc7e0-69d2-4e31-a6de-be9d2a05812f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:39.279549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 20:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:39.426928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf2452-5010-46fa-9d32-3cdda508ec07 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:39.441588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf2452-5010-46fa-9d32-3cdda508ec07 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:39.446930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f4abb-a25d-43dc-ae76-06056c44536b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:39.447562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4abb-a25d-43dc-ae76-06056c44536b service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:39.447848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4abb-a25d-43dc-ae76-06056c44536b service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:39.448128 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4abb-a25d-43dc-ae76-06056c44536b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:39.456670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f4abb-a25d-43dc-ae76-06056c44536b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:39.458254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f4abb-a25d-43dc-ae76-06056c44536b service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:39.467671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 723/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:41.525815 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d101732-6779-4c46-bdcb-8bf3646d3921 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:41.526348 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d101732-6779-4c46-bdcb-8bf3646d3921 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:41.526348 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d101732-6779-4c46-bdcb-8bf3646d3921 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:41.527752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:41.533301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d8bcc-91a6-4d82-816c-6ef73a2c9932 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:41.533722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d8bcc-91a6-4d82-816c-6ef73a2c9932 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:41.533884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d8bcc-91a6-4d82-816c-6ef73a2c9932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:41.551883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-472d8bcc-91a6-4d82-816c-6ef73a2c9932 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:41.553078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-472d8bcc-91a6-4d82-816c-6ef73a2c9932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:41.561453 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:57:41.598494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d101732-6779-4c46-bdcb-8bf3646d3921 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:41.609899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d101732-6779-4c46-bdcb-8bf3646d3921 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:41.613357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54e681-6333-44cb-b99a-761685e3fac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:41.613984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54e681-6333-44cb-b99a-761685e3fac7 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:41.614269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54e681-6333-44cb-b99a-761685e3fac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:41.614577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54e681-6333-44cb-b99a-761685e3fac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:41.624249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54e681-6333-44cb-b99a-761685e3fac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:41.625683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54e681-6333-44cb-b99a-761685e3fac7 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:41.635228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 719/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:44.026040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07935d-b0ac-4625-80ba-449069a7defb None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:44.026611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07935d-b0ac-4625-80ba-449069a7defb None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:44.026611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07935d-b0ac-4625-80ba-449069a7defb None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:44.028517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:44.033699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bf6b1-c52d-4579-a9ce-34ec28798dd9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:44.034158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bf6b1-c52d-4579-a9ce-34ec28798dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:44.034285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bf6b1-c52d-4579-a9ce-34ec28798dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:44.052215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9f0bf6b1-c52d-4579-a9ce-34ec28798dd9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:44.053408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9f0bf6b1-c52d-4579-a9ce-34ec28798dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:44.061100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:44.098566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07935d-b0ac-4625-80ba-449069a7defb None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:44.109512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07935d-b0ac-4625-80ba-449069a7defb None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:44.112990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d07ab4-f1b6-445e-9b73-271838b51393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:44.113616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d07ab4-f1b6-445e-9b73-271838b51393 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:44.113898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d07ab4-f1b6-445e-9b73-271838b51393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:44.114176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d07ab4-f1b6-445e-9b73-271838b51393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:44.122828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d07ab4-f1b6-445e-9b73-271838b51393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:44.124236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d07ab4-f1b6-445e-9b73-271838b51393 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:44.133769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 726/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:46.662718 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5d6b2-3014-4af7-afb0-d97e57ec15a2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:46.663222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5d6b2-3014-4af7-afb0-d97e57ec15a2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:46.663339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5d6b2-3014-4af7-afb0-d97e57ec15a2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:46.664668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:46.670667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d484fe-b5fb-4527-80f1-38e3d170829e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:46.671077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d484fe-b5fb-4527-80f1-38e3d170829e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:46.671228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d484fe-b5fb-4527-80f1-38e3d170829e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:46.689395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c1d484fe-b5fb-4527-80f1-38e3d170829e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:46.690726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c1d484fe-b5fb-4527-80f1-38e3d170829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:46.698407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:46.735802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5d6b2-3014-4af7-afb0-d97e57ec15a2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:46.746939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5d6b2-3014-4af7-afb0-d97e57ec15a2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:46.750418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31309c2-10d1-4937-9b47-f340ed4451c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:46.751084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31309c2-10d1-4937-9b47-f340ed4451c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:46.751372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31309c2-10d1-4937-9b47-f340ed4451c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:46.751653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31309c2-10d1-4937-9b47-f340ed4451c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:46.760611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31309c2-10d1-4937-9b47-f340ed4451c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:46.762038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31309c2-10d1-4937-9b47-f340ed4451c9 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:46.771550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 722/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:49.101576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884eeaf-b43c-4931-b189-6ac99f2a5a9c None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:49.102131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884eeaf-b43c-4931-b189-6ac99f2a5a9c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:49.102131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8884eeaf-b43c-4931-b189-6ac99f2a5a9c None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:49.103654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:49.109431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e72ad-02a5-4f80-bd3f-8df9696b8ab1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:49.109880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e72ad-02a5-4f80-bd3f-8df9696b8ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:49.109983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e72ad-02a5-4f80-bd3f-8df9696b8ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:49.127853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-209e72ad-02a5-4f80-bd3f-8df9696b8ab1 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:49.129069 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-209e72ad-02a5-4f80-bd3f-8df9696b8ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:49.137211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:49.182220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884eeaf-b43c-4931-b189-6ac99f2a5a9c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:49.193475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8884eeaf-b43c-4931-b189-6ac99f2a5a9c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:49.197019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665e653-0e2b-4e5e-8c60-dbd959a373e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:49.197657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665e653-0e2b-4e5e-8c60-dbd959a373e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:49.197945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665e653-0e2b-4e5e-8c60-dbd959a373e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:49.198227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665e653-0e2b-4e5e-8c60-dbd959a373e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:49.207037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665e653-0e2b-4e5e-8c60-dbd959a373e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:49.208712 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665e653-0e2b-4e5e-8c60-dbd959a373e5 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:49.218528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 729/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:51.449319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd74256-d62f-4c79-829f-0d1d344289e2 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:51.449900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd74256-d62f-4c79-829f-0d1d344289e2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:51.449900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd74256-d62f-4c79-829f-0d1d344289e2 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:51.451423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:51.456666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4b7f8-265a-4f00-b166-ee8b92e709e8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:51.457097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4b7f8-265a-4f00-b166-ee8b92e709e8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:51.457233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4b7f8-265a-4f00-b166-ee8b92e709e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:51.474930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-01d4b7f8-265a-4f00-b166-ee8b92e709e8 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:51.476131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-01d4b7f8-265a-4f00-b166-ee8b92e709e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:51.483971 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:51.521557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd74256-d62f-4c79-829f-0d1d344289e2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:51.532084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd74256-d62f-4c79-829f-0d1d344289e2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:51.535513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc3011-6c85-4c9f-9d3c-8a9a73977323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:51.536134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc3011-6c85-4c9f-9d3c-8a9a73977323 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:51.536415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc3011-6c85-4c9f-9d3c-8a9a73977323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:51.536689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc3011-6c85-4c9f-9d3c-8a9a73977323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:51.545252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc3011-6c85-4c9f-9d3c-8a9a73977323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:51.546715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc3011-6c85-4c9f-9d3c-8a9a73977323 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:51.555854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 725/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:53.786846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29469f61-7abd-40d1-b563-2901c83a9618 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:53.787410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29469f61-7abd-40d1-b563-2901c83a9618 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:53.787410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29469f61-7abd-40d1-b563-2901c83a9618 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:53.788779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:53.794720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75b0b4-49fd-41ed-a40a-8d89ac674ec7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:53.795142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75b0b4-49fd-41ed-a40a-8d89ac674ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:53.795285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75b0b4-49fd-41ed-a40a-8d89ac674ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:53.813076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0a75b0b4-49fd-41ed-a40a-8d89ac674ec7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:53.814667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0a75b0b4-49fd-41ed-a40a-8d89ac674ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:53.821897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:53.859652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29469f61-7abd-40d1-b563-2901c83a9618 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:53.870883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29469f61-7abd-40d1-b563-2901c83a9618 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:53.874502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514eb55-58fa-4d7b-93ff-7654c2679f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:53.875158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514eb55-58fa-4d7b-93ff-7654c2679f45 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:53.875441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514eb55-58fa-4d7b-93ff-7654c2679f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:53.875722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514eb55-58fa-4d7b-93ff-7654c2679f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:53.884890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514eb55-58fa-4d7b-93ff-7654c2679f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:53.886318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514eb55-58fa-4d7b-93ff-7654c2679f45 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:53.895634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 732/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:56.129546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3ccac-d203-4c5e-92ef-36a1a311d4d4 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:56.130366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3ccac-d203-4c5e-92ef-36a1a311d4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:56.130366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3ccac-d203-4c5e-92ef-36a1a311d4d4 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:56.131730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:56.137519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720af7-ca5f-481f-b437-5e29f559d42e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:56.137912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720af7-ca5f-481f-b437-5e29f559d42e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:56.138084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720af7-ca5f-481f-b437-5e29f559d42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:56.144261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=70 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:57:56.157002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-98720af7-ca5f-481f-b437-5e29f559d42e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:56.158225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-98720af7-ca5f-481f-b437-5e29f559d42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:56.165964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:56.203523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3ccac-d203-4c5e-92ef-36a1a311d4d4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:56.208930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=73 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:57:56.215364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3ccac-d203-4c5e-92ef-36a1a311d4d4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:56.218895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077e949-c0fc-4bfc-99d5-530ce2647f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:56.219533 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077e949-c0fc-4bfc-99d5-530ce2647f09 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:56.219821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077e949-c0fc-4bfc-99d5-530ce2647f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:56.220105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077e949-c0fc-4bfc-99d5-530ce2647f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:56.229169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077e949-c0fc-4bfc-99d5-530ce2647f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:56.230696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077e949-c0fc-4bfc-99d5-530ce2647f09 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:56.240092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 728/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:57:58.479977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a01e7-9833-4d44-9ff4-f53da3d07e24 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:58.480503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a01e7-9833-4d44-9ff4-f53da3d07e24 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:58.480627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a01e7-9833-4d44-9ff4-f53da3d07e24 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:58.481924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:57:58.487328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b77b8e-b08c-4945-83e9-38008911a842 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:58.487737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b77b8e-b08c-4945-83e9-38008911a842 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:58.487895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b77b8e-b08c-4945-83e9-38008911a842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:58.506024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-00b77b8e-b08c-4945-83e9-38008911a842 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:57:58.507247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-00b77b8e-b08c-4945-83e9-38008911a842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:57:58.515587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:57:58.552042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a01e7-9833-4d44-9ff4-f53da3d07e24 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:57:58.563305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a01e7-9833-4d44-9ff4-f53da3d07e24 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:57:58.566786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e904b33-7720-4142-921a-a07f2701d36a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:57:58.567604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e904b33-7720-4142-921a-a07f2701d36a service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:57:58.567894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e904b33-7720-4142-921a-a07f2701d36a service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:57:58.568168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e904b33-7720-4142-921a-a07f2701d36a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:57:58.576936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e904b33-7720-4142-921a-a07f2701d36a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:57:58.578371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e904b33-7720-4142-921a-a07f2701d36a service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:57:58.588512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 735/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:58:00.785728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0c43-e3f3-4389-abcf-d675e2b8c658 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:00.786288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0c43-e3f3-4389-abcf-d675e2b8c658 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:00.786288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0c43-e3f3-4389-abcf-d675e2b8c658 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:00.787687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:58:00.793585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0b18-d406-482b-ab6c-3dc45fc9e7d6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:00.794004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0b18-d406-482b-ab6c-3dc45fc9e7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:00.794141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc0b18-d406-482b-ab6c-3dc45fc9e7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:00.812660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-20dc0b18-d406-482b-ab6c-3dc45fc9e7d6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:58:00.813848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-20dc0b18-d406-482b-ab6c-3dc45fc9e7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:00.821623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:58:00.857133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a0c43-e3f3-4389-abcf-d675e2b8c658 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:58:00.867701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a0c43-e3f3-4389-abcf-d675e2b8c658 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:58:00.871824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822162bc-94dc-4f21-9c4c-f0a59c24c617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:58:00.872450 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822162bc-94dc-4f21-9c4c-f0a59c24c617 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:00.872734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822162bc-94dc-4f21-9c4c-f0a59c24c617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:00.873010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822162bc-94dc-4f21-9c4c-f0a59c24c617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:00.882397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822162bc-94dc-4f21-9c4c-f0a59c24c617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:58:00.883914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822162bc-94dc-4f21-9c4c-f0a59c24c617 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:58:00.893318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 731/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:58:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:58:03.210057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a7561-5e6f-4d5f-8573-ef7124fe66e8 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:03.210617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a7561-5e6f-4d5f-8573-ef7124fe66e8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:03.210617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044a7561-5e6f-4d5f-8573-ef7124fe66e8 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:03.212007 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 20:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:58:03.217821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381a36-4179-41cc-b6c7-0bf7ae03ff18 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:03.218247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381a36-4179-41cc-b6c7-0bf7ae03ff18 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:03.218362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381a36-4179-41cc-b6c7-0bf7ae03ff18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:03.235781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-88381a36-4179-41cc-b6c7-0bf7ae03ff18 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:58:03.236972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-88381a36-4179-41cc-b6c7-0bf7ae03ff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:03.244712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 20:58:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:58:03.281961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a7561-5e6f-4d5f-8573-ef7124fe66e8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:58:03.283636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-044a7561-5e6f-4d5f-8573-ef7124fe66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:03.296295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044a7561-5e6f-4d5f-8573-ef7124fe66e8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:58:03.299738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34399959-a1c5-41d1-b18f-8b1b081a8529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:58:03.300364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34399959-a1c5-41d1-b18f-8b1b081a8529 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:03.300648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34399959-a1c5-41d1-b18f-8b1b081a8529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:03.300923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34399959-a1c5-41d1-b18f-8b1b081a8529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:03.309679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34399959-a1c5-41d1-b18f-8b1b081a8529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:58:03.311177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34399959-a1c5-41d1-b18f-8b1b081a8529 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:58:03.320596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 738/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:58:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:58:13.230847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=17 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:58:13.316468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=17 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:58:36.240919 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ec2a2-973b-4997-aba1-6fe91f90cb13 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:36.241492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ec2a2-973b-4997-aba1-6fe91f90cb13 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:36.241492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ec2a2-973b-4997-aba1-6fe91f90cb13 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:36.242839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 733/1471] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:58:36.247863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af60dc-38d7-47f4-a4c9-aee5bd2101a2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:36.248264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af60dc-38d7-47f4-a4c9-aee5bd2101a2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:36.248420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af60dc-38d7-47f4-a4c9-aee5bd2101a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:36.266424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-06af60dc-38d7-47f4-a4c9-aee5bd2101a2 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:58:36.267631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-06af60dc-38d7-47f4-a4c9-aee5bd2101a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:36.275227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 739/1472] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:58:36.320694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b45704-ea9f-4199-958c-d9727081fcb3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:36.320982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b45704-ea9f-4199-958c-d9727081fcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:36.321256 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b45704-ea9f-4199-958c-d9727081fcb3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:36.323145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 734/1473] 10.222.0.57 () {54 vars in 962 bytes} [Thu May 26 20:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:58:36.327856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79a9b3-d9f8-45f7-ad75-153a78afe86e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:36.328153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79a9b3-d9f8-45f7-ad75-153a78afe86e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:36.328434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac79a9b3-d9f8-45f7-ad75-153a78afe86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:36.346276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ac79a9b3-d9f8-45f7-ad75-153a78afe86e None None] MFA Rules not processed for user `2c3bd8dd462846bf9423f8528aa5c489`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:58:36.347468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ac79a9b3-d9f8-45f7-ad75-153a78afe86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:36.353728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 740/1474] 10.222.0.57 () {58 vars in 1059 bytes} [Thu May 26 20:58:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 20:58:36.360640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b45704-ea9f-4199-958c-d9727081fcb3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:58:36.370877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b45704-ea9f-4199-958c-d9727081fcb3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:58:36.374264 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3ae51-3b06-45a9-adf4-2a8f5da38d2a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3bd8dd462846bf9423f8528aa5c489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:58:36.374909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3ae51-3b06-45a9-adf4-2a8f5da38d2a service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:36.375191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3ae51-3b06-45a9-adf4-2a8f5da38d2a service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:36.375463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3ae51-3b06-45a9-adf4-2a8f5da38d2a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:36.384428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3ae51-3b06-45a9-adf4-2a8f5da38d2a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:58:36.386333 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3ae51-3b06-45a9-adf4-2a8f5da38d2a service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:58:36.395607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 735/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:58:38.655728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5feb4a-45d8-4ccf-8d78-9b6a06aa26da None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:38.656235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5feb4a-45d8-4ccf-8d78-9b6a06aa26da None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:38.656366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5feb4a-45d8-4ccf-8d78-9b6a06aa26da None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:38.657653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 741/1476] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:58:38.663031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87920c94-efd0-47f8-abf3-b36add7f9cad None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:38.663474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87920c94-efd0-47f8-abf3-b36add7f9cad None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:38.663575 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87920c94-efd0-47f8-abf3-b36add7f9cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:38.681500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-87920c94-efd0-47f8-abf3-b36add7f9cad None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:58:38.683341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-87920c94-efd0-47f8-abf3-b36add7f9cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:38.691588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 736/1477] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:58:38.714322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5feb4a-45d8-4ccf-8d78-9b6a06aa26da None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:58:38.726436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5feb4a-45d8-4ccf-8d78-9b6a06aa26da None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:58:38.731496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b0148-5841-4e2b-9afe-a20f120089b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:58:38.732129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b0148-5841-4e2b-9afe-a20f120089b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:38.732439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b0148-5841-4e2b-9afe-a20f120089b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:38.732713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b0148-5841-4e2b-9afe-a20f120089b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:38.733435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b0148-5841-4e2b-9afe-a20f120089b1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:58:38.735237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-629b0148-5841-4e2b-9afe-a20f120089b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 20:58:38.735237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: warnings.warn(msg) May 26 20:58:38.735237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: May 26 20:58:38.736407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b0148-5841-4e2b-9afe-a20f120089b1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:58:38.755580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 742/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Thu May 26 20:58:38 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 20:58:40.968117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e780e7-1456-4c00-adef-8ba31683689e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:40.968917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e780e7-1456-4c00-adef-8ba31683689e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:40.968917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e780e7-1456-4c00-adef-8ba31683689e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:40.969907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 737/1479] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:58:40.975306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640780e4-963c-49f1-a4fd-a3a086bfdd15 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:40.975754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640780e4-963c-49f1-a4fd-a3a086bfdd15 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:40.975852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640780e4-963c-49f1-a4fd-a3a086bfdd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:40.994775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-640780e4-963c-49f1-a4fd-a3a086bfdd15 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:58:40.995990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-640780e4-963c-49f1-a4fd-a3a086bfdd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:41.003715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 743/1480] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:58:41.048228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e780e7-1456-4c00-adef-8ba31683689e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:58:41.062423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e780e7-1456-4c00-adef-8ba31683689e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:58:41.067716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2ee34-0ff7-45fb-be4a-9985e8ef8738 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:58:41.068344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2ee34-0ff7-45fb-be4a-9985e8ef8738 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:41.068627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2ee34-0ff7-45fb-be4a-9985e8ef8738 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:41.068904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2ee34-0ff7-45fb-be4a-9985e8ef8738 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:41.077017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2ee34-0ff7-45fb-be4a-9985e8ef8738 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:58:41.078676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2ee34-0ff7-45fb-be4a-9985e8ef8738 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:58:41.088710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 738/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:58:43.445866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9db7-1114-48f0-9779-3d8e57156922 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:43.446406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9db7-1114-48f0-9779-3d8e57156922 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:43.446406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9db7-1114-48f0-9779-3d8e57156922 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:43.448477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 744/1482] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:58:43.453748 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be124c-9f01-4ec2-a3f8-c0799236bd4b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:43.454147 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be124c-9f01-4ec2-a3f8-c0799236bd4b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:43.454304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be124c-9f01-4ec2-a3f8-c0799236bd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:43.471901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-31be124c-9f01-4ec2-a3f8-c0799236bd4b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:58:43.473124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-31be124c-9f01-4ec2-a3f8-c0799236bd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:43.480854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 739/1483] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:58:43.517532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f9db7-1114-48f0-9779-3d8e57156922 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:58:43.528271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f9db7-1114-48f0-9779-3d8e57156922 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:58:43.531793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f1c31-5dfe-4432-b7aa-7e43847b56f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:58:43.532426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f1c31-5dfe-4432-b7aa-7e43847b56f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:43.532714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f1c31-5dfe-4432-b7aa-7e43847b56f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:43.532992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f1c31-5dfe-4432-b7aa-7e43847b56f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:43.542254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f1c31-5dfe-4432-b7aa-7e43847b56f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:58:43.543717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f1c31-5dfe-4432-b7aa-7e43847b56f5 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:58:43.553035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 745/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:58:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:58:45.785708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98364927-e9ee-44f9-8535-d69de083afae None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:45.786246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98364927-e9ee-44f9-8535-d69de083afae None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:45.786370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98364927-e9ee-44f9-8535-d69de083afae None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:45.787671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 740/1485] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:58:45.793528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b7fb3-98fd-45af-9cc0-780e6fee85bc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:45.793934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b7fb3-98fd-45af-9cc0-780e6fee85bc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:45.794080 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b7fb3-98fd-45af-9cc0-780e6fee85bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:45.813347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e33b7fb3-98fd-45af-9cc0-780e6fee85bc None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:58:45.815026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e33b7fb3-98fd-45af-9cc0-780e6fee85bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:45.822291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 746/1486] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:58:45.859288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98364927-e9ee-44f9-8535-d69de083afae None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:58:45.870500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98364927-e9ee-44f9-8535-d69de083afae None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:58:45.873971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b10778-6fdc-4531-9901-bc6a0c1b186b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:58:45.874633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b10778-6fdc-4531-9901-bc6a0c1b186b service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:45.874922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b10778-6fdc-4531-9901-bc6a0c1b186b service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:45.875201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b10778-6fdc-4531-9901-bc6a0c1b186b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:45.884149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b10778-6fdc-4531-9901-bc6a0c1b186b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:58:45.885575 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b10778-6fdc-4531-9901-bc6a0c1b186b service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:58:45.894824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 741/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:58:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:58:48.157517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94043b-e9e7-4e28-8ac8-1b61abd4d557 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:48.158031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94043b-e9e7-4e28-8ac8-1b61abd4d557 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:48.158031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94043b-e9e7-4e28-8ac8-1b61abd4d557 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:48.159395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 747/1488] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:58:48.164584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71355-67ac-41aa-a3b6-fe05f5128838 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:48.164984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71355-67ac-41aa-a3b6-fe05f5128838 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:48.165154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf71355-67ac-41aa-a3b6-fe05f5128838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:48.182672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ecf71355-67ac-41aa-a3b6-fe05f5128838 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:58:48.183874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ecf71355-67ac-41aa-a3b6-fe05f5128838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:48.191924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 742/1489] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:58:48.232431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94043b-e9e7-4e28-8ac8-1b61abd4d557 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:58:48.242955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94043b-e9e7-4e28-8ac8-1b61abd4d557 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:58:48.246362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af7b21-224c-4bca-9b7f-8d03dd62f3ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:58:48.247039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af7b21-224c-4bca-9b7f-8d03dd62f3ed service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:48.247335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af7b21-224c-4bca-9b7f-8d03dd62f3ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:48.247590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af7b21-224c-4bca-9b7f-8d03dd62f3ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:48.256498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af7b21-224c-4bca-9b7f-8d03dd62f3ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:58:48.257903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af7b21-224c-4bca-9b7f-8d03dd62f3ed service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:58:48.267542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 748/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:58:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:58:50.653084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbb785-9a1a-4f36-b688-7092011b58c3 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:50.653599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbb785-9a1a-4f36-b688-7092011b58c3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:50.653599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efbb785-9a1a-4f36-b688-7092011b58c3 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:50.654976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 743/1491] 10.222.0.57 () {54 vars in 926 bytes} [Thu May 26 20:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 20:58:50.660124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94919464-dcef-4450-9fcf-213716b4a577 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:50.660557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94919464-dcef-4450-9fcf-213716b4a577 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:50.660660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94919464-dcef-4450-9fcf-213716b4a577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:50.677838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-94919464-dcef-4450-9fcf-213716b4a577 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 20:58:50.679062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-94919464-dcef-4450-9fcf-213716b4a577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 20:58:50.687226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 749/1492] 10.222.0.57 () {58 vars in 1023 bytes} [Thu May 26 20:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 20:58:50.725055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbb785-9a1a-4f36-b688-7092011b58c3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 20:58:50.735651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efbb785-9a1a-4f36-b688-7092011b58c3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 20:58:50.739126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142cc2bf-6fd1-43fe-8100-f4e5fd7f8099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 20:58:50.739751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cc2bf-6fd1-43fe-8100-f4e5fd7f8099 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 20:58:50.740036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cc2bf-6fd1-43fe-8100-f4e5fd7f8099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 20:58:50.740319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142cc2bf-6fd1-43fe-8100-f4e5fd7f8099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 20:58:50.749453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cc2bf-6fd1-43fe-8100-f4e5fd7f8099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 20:58:50.750983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142cc2bf-6fd1-43fe-8100-f4e5fd7f8099 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 20:58:50.761239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 744/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 20:58:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 20:59:00.672814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=33 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 20:59:00.756310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=33 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:00:36.426033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5f46a-0c52-4917-a60f-b3b379ab6819 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.426641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5f46a-0c52-4917-a60f-b3b379ab6819 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.426641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5f46a-0c52-4917-a60f-b3b379ab6819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.445434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-18d5f46a-0c52-4917-a60f-b3b379ab6819 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:00:36.446669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-18d5f46a-0c52-4917-a60f-b3b379ab6819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:00:36.457362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 750/1494] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:00:36.466840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd2c3f-d5b8-432f-a2a6-f2efb5792dfb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.467341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd2c3f-d5b8-432f-a2a6-f2efb5792dfb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.467476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd2c3f-d5b8-432f-a2a6-f2efb5792dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.485513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-93dd2c3f-d5b8-432f-a2a6-f2efb5792dfb None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:00:36.486720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-93dd2c3f-d5b8-432f-a2a6-f2efb5792dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:00:36.494903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 745/1495] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:00:36.499390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5f46a-0c52-4917-a60f-b3b379ab6819 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:00:36.509761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5f46a-0c52-4917-a60f-b3b379ab6819 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:00:36.514282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba8982-2504-4606-b7af-b47a6b11b116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:00:36.514977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba8982-2504-4606-b7af-b47a6b11b116 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.515261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba8982-2504-4606-b7af-b47a6b11b116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.515533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba8982-2504-4606-b7af-b47a6b11b116 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.523708 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba8982-2504-4606-b7af-b47a6b11b116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:00:36.524025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba8982-2504-4606-b7af-b47a6b11b116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:00:36.525106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-80ba8982-2504-4606-b7af-b47a6b11b116 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 21:00:36.525106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: warnings.warn(msg) May 26 21:00:36.525106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: May 26 21:00:36.526248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba8982-2504-4606-b7af-b47a6b11b116 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:00:36.545165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 751/1496] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:00:36.553377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd2c3f-d5b8-432f-a2a6-f2efb5792dfb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:00:36.563100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd2c3f-d5b8-432f-a2a6-f2efb5792dfb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:00:36.567647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec0ac6-4899-4749-9b5c-5e54e11359db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:00:36.568295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0ac6-4899-4749-9b5c-5e54e11359db admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.568581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0ac6-4899-4749-9b5c-5e54e11359db admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.568859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec0ac6-4899-4749-9b5c-5e54e11359db admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.569670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec0ac6-4899-4749-9b5c-5e54e11359db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:00:36.571976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-31ec0ac6-4899-4749-9b5c-5e54e11359db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:00:36.571976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 26 21:00:36.571976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: May 26 21:00:36.573173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec0ac6-4899-4749-9b5c-5e54e11359db admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:00:36.608893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 746/1497] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:00:36 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:00:36.613646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba8982-2504-4606-b7af-b47a6b11b116 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:00:36.624928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba8982-2504-4606-b7af-b47a6b11b116 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:00:36.679998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea8a7c-d97c-4011-af05-13e19f95f730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:00:36.680626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea8a7c-d97c-4011-af05-13e19f95f730 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.680913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea8a7c-d97c-4011-af05-13e19f95f730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.681176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea8a7c-d97c-4011-af05-13e19f95f730 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.681863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea8a7c-d97c-4011-af05-13e19f95f730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:00:36.682922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-3bea8a7c-d97c-4011-af05-13e19f95f730 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:00:36.682922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: warnings.warn(msg) May 26 21:00:36.682922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: May 26 21:00:36.684122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bea8a7c-d97c-4011-af05-13e19f95f730 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:00:36.703936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 752/1498] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:00:36 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:00:36.708044 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec0ac6-4899-4749-9b5c-5e54e11359db admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:00:36.718020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec0ac6-4899-4749-9b5c-5e54e11359db admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:00:36.722461 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0da48-0a78-478f-8f3c-260d4c7e757b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:00:36.723116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0da48-0a78-478f-8f3c-260d4c7e757b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.723399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0da48-0a78-478f-8f3c-260d4c7e757b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.723674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0da48-0a78-478f-8f3c-260d4c7e757b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.724387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0da48-0a78-478f-8f3c-260d4c7e757b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:00:36.725399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-86c0da48-0a78-478f-8f3c-260d4c7e757b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 21:00:36.725399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 26 21:00:36.725399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: May 26 21:00:36.726266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0da48-0a78-478f-8f3c-260d4c7e757b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:00:36.738587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 747/1499] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:00:36.742950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea8a7c-d97c-4011-af05-13e19f95f730 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:00:36.752536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bea8a7c-d97c-4011-af05-13e19f95f730 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:00:36.758009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a5b75-2ba4-4b9c-be42-9a04ee7d14a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:00:36.758727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a5b75-2ba4-4b9c-be42-9a04ee7d14a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.759014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a5b75-2ba4-4b9c-be42-9a04ee7d14a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.759293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a5b75-2ba4-4b9c-be42-9a04ee7d14a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.760010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a5b75-2ba4-4b9c-be42-9a04ee7d14a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:00:36.761298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-ae4a5b75-2ba4-4b9c-be42-9a04ee7d14a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 21:00:36.761298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: warnings.warn(msg) May 26 21:00:36.761298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: May 26 21:00:36.762201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a5b75-2ba4-4b9c-be42-9a04ee7d14a3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:00:36.775819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 753/1500] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:00:36.779663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0da48-0a78-478f-8f3c-260d4c7e757b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:00:36.790039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0da48-0a78-478f-8f3c-260d4c7e757b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:00:36.794779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc18d38-479a-4863-b414-0230820e4b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:00:36.795653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc18d38-479a-4863-b414-0230820e4b4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.795965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc18d38-479a-4863-b414-0230820e4b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.796247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc18d38-479a-4863-b414-0230820e4b4e admin admin] PATH_INFO: `/v3/projects/37c91492c7a745999669160810f6fe9f/users/c1689f7e999d42a5a2072f3baa3132ee/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.811702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc18d38-479a-4863-b414-0230820e4b4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=37c91492c7a745999669160810f6fe9f, user_id=c1689f7e999d42a5a2072f3baa3132ee, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:00:36.812754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-bfc18d38-479a-4863-b414-0230820e4b4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:00:36.812754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 26 21:00:36.812754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: May 26 21:00:36.815137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc18d38-479a-4863-b414-0230820e4b4e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:00:36.827445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 748/1501] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:00:36 2022] PUT /identity/v3/projects/37c91492c7a745999669160810f6fe9f/users/c1689f7e999d42a5a2072f3baa3132ee/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:00:36.838599 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2127f-6fc7-4e34-bae3-485924e36ead None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.838892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2127f-6fc7-4e34-bae3-485924e36ead None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.839172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2127f-6fc7-4e34-bae3-485924e36ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.867469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-baa2127f-6fc7-4e34-bae3-485924e36ead None None] MFA Rules not processed for user `c1689f7e999d42a5a2072f3baa3132ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:00:36.868884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-baa2127f-6fc7-4e34-bae3-485924e36ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:00:36.931648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 754/1502] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:00:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:00:36.944680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a485ac-8a66-48ce-a8fa-1d533c4af797 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.944973 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a485ac-8a66-48ce-a8fa-1d533c4af797 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.945254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a485ac-8a66-48ce-a8fa-1d533c4af797 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.946664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 749/1503] 10.222.0.57 () {54 vars in 963 bytes} [Thu May 26 21:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:00:36.951366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59af76-145f-44d1-a72b-47319b52a058 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:36.951661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59af76-145f-44d1-a72b-47319b52a058 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:36.951944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59af76-145f-44d1-a72b-47319b52a058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:36.970027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6f59af76-145f-44d1-a72b-47319b52a058 None None] MFA Rules not processed for user `afe351334485400a8c11f17492c9ff5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:00:36.971319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6f59af76-145f-44d1-a72b-47319b52a058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:00:37.013966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:00:37.022778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 755/1504] 10.222.0.57 () {58 vars in 1060 bytes} [Thu May 26 21:00:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:00:37.030176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a485ac-8a66-48ce-a8fa-1d533c4af797 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:00:37.041212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a485ac-8a66-48ce-a8fa-1d533c4af797 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:00:37.044837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cf40d-c1e3-420c-8ad0-bf884d141c75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:00:37.045471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf40d-c1e3-420c-8ad0-bf884d141c75 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:37.045758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf40d-c1e3-420c-8ad0-bf884d141c75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:37.046036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf40d-c1e3-420c-8ad0-bf884d141c75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:37.055182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cf40d-c1e3-420c-8ad0-bf884d141c75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:00:37.056619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cf40d-c1e3-420c-8ad0-bf884d141c75 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:00:37.065569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 750/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:00:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:00:37.106530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd88fc-7f11-4996-81d9-36879bf8b298 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:37.106823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd88fc-7f11-4996-81d9-36879bf8b298 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:37.107104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd88fc-7f11-4996-81d9-36879bf8b298 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:37.108390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 756/1506] 10.222.0.57 () {52 vars in 829 bytes} [Thu May 26 21:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:00:37.111885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cf40d-c1e3-420c-8ad0-bf884d141c75 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:00:37.173041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cf40d-c1e3-420c-8ad0-bf884d141c75 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:00:37.183500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:00:37.184313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:37.184598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:37.184875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] PATH_INFO: `/v3/users/c1689f7e999d42a5a2072f3baa3132ee` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:37.186065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1689f7e999d42a5a2072f3baa3132ee)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:00:37.187130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:00:37.187130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 26 21:00:37.187130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: May 26 21:00:37.188327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:00:37.251875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1689f7e999d42a5a2072f3baa3132ee'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:00:37.252515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1689f7e999d42a5a2072f3baa3132ee'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:00:37.261568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1689f7e999d42a5a2072f3baa3132ee'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:00:37.262302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1689f7e999d42a5a2072f3baa3132ee'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:00:37.281171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1689f7e999d42a5a2072f3baa3132ee'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:00:37.289994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:00:37.301245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 751/1507] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:00:37 2022] DELETE /identity/v3/users/c1689f7e999d42a5a2072f3baa3132ee => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:00:37.305662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361df4a-a64d-4eef-80d8-f07dba2d215f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:37.305953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361df4a-a64d-4eef-80d8-f07dba2d215f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:37.306236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361df4a-a64d-4eef-80d8-f07dba2d215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:37.323457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1361df4a-a64d-4eef-80d8-f07dba2d215f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:00:37.324646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1361df4a-a64d-4eef-80d8-f07dba2d215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:00:37.382185 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 757/1508] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:00:37.392671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:00:37.394085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:00:37.449742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5975271-f200-4cf0-95f1-8a711ba65a41 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:00:37.453255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75d3ab-659e-4e3c-806c-63d04be4545f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:00:37.453891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75d3ab-659e-4e3c-806c-63d04be4545f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:37.454177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75d3ab-659e-4e3c-806c-63d04be4545f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:37.454472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75d3ab-659e-4e3c-806c-63d04be4545f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:37.463833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75d3ab-659e-4e3c-806c-63d04be4545f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:00:37.465281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75d3ab-659e-4e3c-806c-63d04be4545f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:00:37.474564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 752/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:00:37.797634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361df4a-a64d-4eef-80d8-f07dba2d215f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:00:37.799187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1361df4a-a64d-4eef-80d8-f07dba2d215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:00:37.813901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361df4a-a64d-4eef-80d8-f07dba2d215f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:00:37.820239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:00:37.820698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:00:37.820981 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:00:37.821259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] PATH_INFO: `/v3/projects/37c91492c7a745999669160810f6fe9f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:00:37.822672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=37c91492c7a745999669160810f6fe9f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:00:37.823842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:00:37.823842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: warnings.warn(msg) May 26 21:00:37.823842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: May 26 21:00:37.825068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:00:37.886134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c91492c7a745999669160810f6fe9f', 'request_id': 'req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:00:37.899685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c91492c7a745999669160810f6fe9f', 'request_id': 'req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:00:37.905655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c91492c7a745999669160810f6fe9f', 'request_id': 'req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:00:37.920655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] The token cache is being invalidate because project 37c91492c7a745999669160810f6fe9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:00:37.920942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:00:37.921558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9ffeab91-8c0e-4ba5-8837-ea0ddee1dc9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:00:37.923241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 758/1510] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:00:37 2022] DELETE /identity/v3/projects/37c91492c7a745999669160810f6fe9f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:00:47.470435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=29 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:00:47.917417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:02:18.364933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8049f9-b822-47ad-bc4a-18a7415ee00e None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:18.365525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8049f9-b822-47ad-bc4a-18a7415ee00e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:18.365525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8049f9-b822-47ad-bc4a-18a7415ee00e None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:18.367415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:02:18.372327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe15f38-7f10-4f51-95c5-f79d3fcbee14 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:18.372762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe15f38-7f10-4f51-95c5-f79d3fcbee14 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:18.372895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe15f38-7f10-4f51-95c5-f79d3fcbee14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:18.391993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2fe15f38-7f10-4f51-95c5-f79d3fcbee14 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:02:18.393196 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2fe15f38-7f10-4f51-95c5-f79d3fcbee14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:18.450282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:02:18.646728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8049f9-b822-47ad-bc4a-18a7415ee00e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:02:18.648416 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2f8049f9-b822-47ad-bc4a-18a7415ee00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:18.719934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8049f9-b822-47ad-bc4a-18a7415ee00e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:02:18.725825 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63574e5a-cefa-42f5-aa00-500934061f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:02:18.726527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574e5a-cefa-42f5-aa00-500934061f07 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:18.726817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574e5a-cefa-42f5-aa00-500934061f07 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:18.727097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63574e5a-cefa-42f5-aa00-500934061f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:18.735242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63574e5a-cefa-42f5-aa00-500934061f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:02:18.736845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63574e5a-cefa-42f5-aa00-500934061f07 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:02:18.746620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 754/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:02:19.125284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e86e1-d7e8-4c69-b1de-df3668d6ca02 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:19.125582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e86e1-d7e8-4c69-b1de-df3668d6ca02 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:19.125869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e86e1-d7e8-4c69-b1de-df3668d6ca02 None None] PATH_INFO: `/v3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:19.127502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 760/1514] 10.222.0.62 () {54 vars in 925 bytes} [Thu May 26 21:02:19 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:02:19.134084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6423861-e09a-4a7d-bb3f-32b3042847a9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:19.134386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6423861-e09a-4a7d-bb3f-32b3042847a9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:19.134702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6423861-e09a-4a7d-bb3f-32b3042847a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:19.153401 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c6423861-e09a-4a7d-bb3f-32b3042847a9 None None] MFA Rules not processed for user `4edb53dc871047ce89ebb3b90143a6de`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:02:19.154646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c6423861-e09a-4a7d-bb3f-32b3042847a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:19.219391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 755/1515] 10.222.0.62 () {58 vars in 1015 bytes} [Thu May 26 21:02:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:02:19.622250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb183-8587-4afd-a909-7ad6b1ca0818 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:19.622837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb183-8587-4afd-a909-7ad6b1ca0818 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:19.622956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffb183-8587-4afd-a909-7ad6b1ca0818 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:19.624242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 761/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:02:19.629161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363a5ca-3a47-4711-9c58-c5d16740bbd4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:19.629605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363a5ca-3a47-4711-9c58-c5d16740bbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:19.629732 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363a5ca-3a47-4711-9c58-c5d16740bbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:19.648952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e363a5ca-3a47-4711-9c58-c5d16740bbd4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:02:19.650059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e363a5ca-3a47-4711-9c58-c5d16740bbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:19.657886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 756/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:02:19.808299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffb183-8587-4afd-a909-7ad6b1ca0818 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:02:19.823076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffb183-8587-4afd-a909-7ad6b1ca0818 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:02:19.828356 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f67dac-820a-4151-b05d-583683743dd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:02:19.828989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f67dac-820a-4151-b05d-583683743dd0 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:19.829273 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f67dac-820a-4151-b05d-583683743dd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:19.829546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f67dac-820a-4151-b05d-583683743dd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:19.838073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f67dac-820a-4151-b05d-583683743dd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:02:19.839684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f67dac-820a-4151-b05d-583683743dd0 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:02:19.848858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 762/1518] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:02:27.963420 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be1e81-0a1c-4641-93cd-3e5e9b14e2aa None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:27.963954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be1e81-0a1c-4641-93cd-3e5e9b14e2aa None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:27.963954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be1e81-0a1c-4641-93cd-3e5e9b14e2aa None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:27.965265 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 757/1519] 10.222.0.57 () {54 vars in 960 bytes} [Thu May 26 21:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:02:27.970209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80350dd-5be8-4ca6-9aae-16a509a6be00 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:27.970692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80350dd-5be8-4ca6-9aae-16a509a6be00 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:27.970799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80350dd-5be8-4ca6-9aae-16a509a6be00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:27.988960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f80350dd-5be8-4ca6-9aae-16a509a6be00 None None] MFA Rules not processed for user `4edb53dc871047ce89ebb3b90143a6de`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:02:27.990160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f80350dd-5be8-4ca6-9aae-16a509a6be00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:27.999229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 763/1520] 10.222.0.57 () {58 vars in 1057 bytes} [Thu May 26 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:02:28.006884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be1e81-0a1c-4641-93cd-3e5e9b14e2aa None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:02:28.020018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be1e81-0a1c-4641-93cd-3e5e9b14e2aa None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:02:28.025036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d5ee3-c608-47c8-bb02-b3f542a19653 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:02:28.025659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d5ee3-c608-47c8-bb02-b3f542a19653 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:28.025944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d5ee3-c608-47c8-bb02-b3f542a19653 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:28.026220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d5ee3-c608-47c8-bb02-b3f542a19653 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:28.034634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d5ee3-c608-47c8-bb02-b3f542a19653 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:02:28.036241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d5ee3-c608-47c8-bb02-b3f542a19653 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:02:28.046202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 758/1521] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:02:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:02:31.129180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c67a1b-957a-4652-9012-02f441108f63 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:31.129740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c67a1b-957a-4652-9012-02f441108f63 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:31.129740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c67a1b-957a-4652-9012-02f441108f63 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:31.131163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:02:31.135856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0cb09-5bd2-43f8-8084-ed74d10eb562 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:31.136257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0cb09-5bd2-43f8-8084-ed74d10eb562 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:31.136423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0cb09-5bd2-43f8-8084-ed74d10eb562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:31.154463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-09c0cb09-5bd2-43f8-8084-ed74d10eb562 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:02:31.155664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-09c0cb09-5bd2-43f8-8084-ed74d10eb562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:31.163965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:02:31.314850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c67a1b-957a-4652-9012-02f441108f63 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:02:31.329349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c67a1b-957a-4652-9012-02f441108f63 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:02:31.335444 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295665a-6b02-4df9-8b6a-cb7603a38b85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:02:31.336089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295665a-6b02-4df9-8b6a-cb7603a38b85 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:31.336375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295665a-6b02-4df9-8b6a-cb7603a38b85 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:31.336655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295665a-6b02-4df9-8b6a-cb7603a38b85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:31.344848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295665a-6b02-4df9-8b6a-cb7603a38b85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:02:31.346464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295665a-6b02-4df9-8b6a-cb7603a38b85 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:02:31.356375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 765/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:02:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:02:32.208560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb004357-b84e-41b6-9610-9209afdc0b37 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:32.209582 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb004357-b84e-41b6-9610-9209afdc0b37 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:32.209582 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb004357-b84e-41b6-9610-9209afdc0b37 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:32.210919 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:02:32.215680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0ee0-e698-4218-a180-8c4284a185e5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:32.216120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0ee0-e698-4218-a180-8c4284a185e5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:32.216247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f0ee0-e698-4218-a180-8c4284a185e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:32.234573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d51f0ee0-e698-4218-a180-8c4284a185e5 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:02:32.235775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d51f0ee0-e698-4218-a180-8c4284a185e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:32.245254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:02:32.394182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb004357-b84e-41b6-9610-9209afdc0b37 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:02:32.408801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb004357-b84e-41b6-9610-9209afdc0b37 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:02:32.414079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa527f49-a507-4339-a397-1ac95ad3159a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:02:32.414808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa527f49-a507-4339-a397-1ac95ad3159a service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:32.415092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa527f49-a507-4339-a397-1ac95ad3159a service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:32.415370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa527f49-a507-4339-a397-1ac95ad3159a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:32.423280 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa527f49-a507-4339-a397-1ac95ad3159a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:02:32.424858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa527f49-a507-4339-a397-1ac95ad3159a service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:02:32.434094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 761/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:02:36.706765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af345d08-230e-4a0d-ae25-35cb908c1e3a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:36.707309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af345d08-230e-4a0d-ae25-35cb908c1e3a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:36.707435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af345d08-230e-4a0d-ae25-35cb908c1e3a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:36.708788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:02:36.713837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fb2cc-6b79-4eed-b987-4f35d09a147b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:36.714310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fb2cc-6b79-4eed-b987-4f35d09a147b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:36.714415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fb2cc-6b79-4eed-b987-4f35d09a147b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:36.733022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-812fb2cc-6b79-4eed-b987-4f35d09a147b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:02:36.734215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-812fb2cc-6b79-4eed-b987-4f35d09a147b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:36.742695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:02:36.778299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af345d08-230e-4a0d-ae25-35cb908c1e3a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:02:36.779979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-af345d08-230e-4a0d-ae25-35cb908c1e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:36.837989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af345d08-230e-4a0d-ae25-35cb908c1e3a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:02:36.841589 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09a2ab-d0c4-418c-a496-0f0055e1d19e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:02:36.842234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09a2ab-d0c4-418c-a496-0f0055e1d19e service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:36.842555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09a2ab-d0c4-418c-a496-0f0055e1d19e service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:36.842841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09a2ab-d0c4-418c-a496-0f0055e1d19e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:36.852176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09a2ab-d0c4-418c-a496-0f0055e1d19e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:02:36.853621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09a2ab-d0c4-418c-a496-0f0055e1d19e service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:02:36.863095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 768/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:02:39.065981 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4cb48-838a-421a-9b6e-c76db2080220 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:39.066549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4cb48-838a-421a-9b6e-c76db2080220 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:39.066656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4cb48-838a-421a-9b6e-c76db2080220 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:39.068460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:02:39.073670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19a30d-7a9e-444b-b0df-a40b173dc1f5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:39.074080 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19a30d-7a9e-444b-b0df-a40b173dc1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:39.074225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19a30d-7a9e-444b-b0df-a40b173dc1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:39.092637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5d19a30d-7a9e-444b-b0df-a40b173dc1f5 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:02:39.093858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5d19a30d-7a9e-444b-b0df-a40b173dc1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:39.102113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:02:39.137372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4cb48-838a-421a-9b6e-c76db2080220 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:02:39.147965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4cb48-838a-421a-9b6e-c76db2080220 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:02:39.151464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c30d20-cde0-4a48-935b-23f7db308a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:02:39.152086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30d20-cde0-4a48-935b-23f7db308a84 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:39.152367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30d20-cde0-4a48-935b-23f7db308a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:39.152643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c30d20-cde0-4a48-935b-23f7db308a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:39.162426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c30d20-cde0-4a48-935b-23f7db308a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:02:39.163931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c30d20-cde0-4a48-935b-23f7db308a84 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:02:39.173127 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 764/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:02:42.933237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b8599-2364-4636-ba45-bab9efe468a2 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:42.933819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b8599-2364-4636-ba45-bab9efe468a2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:42.933819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b8599-2364-4636-ba45-bab9efe468a2 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:42.935249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:02:42.940412 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79663f33-9bcd-43e0-9386-a6644be5358f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:42.940829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79663f33-9bcd-43e0-9386-a6644be5358f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:42.940970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79663f33-9bcd-43e0-9386-a6644be5358f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:42.958842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-79663f33-9bcd-43e0-9386-a6644be5358f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:02:42.960038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-79663f33-9bcd-43e0-9386-a6644be5358f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:42.967753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:02:43.234950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b8599-2364-4636-ba45-bab9efe468a2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:02:43.236643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-689b8599-2364-4636-ba45-bab9efe468a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:43.292461 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b8599-2364-4636-ba45-bab9efe468a2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:02:43.296063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22126f07-1496-4d5d-8831-9810591103d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:02:43.296706 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22126f07-1496-4d5d-8831-9810591103d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:43.296992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22126f07-1496-4d5d-8831-9810591103d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:43.297271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22126f07-1496-4d5d-8831-9810591103d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:43.306421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22126f07-1496-4d5d-8831-9810591103d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:02:43.307869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22126f07-1496-4d5d-8831-9810591103d7 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:02:43.312751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:02:43.318575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 771/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:02:45.506277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dc526-61fa-4c62-b59b-cfd40b615713 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:45.506901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dc526-61fa-4c62-b59b-cfd40b615713 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:45.506901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400dc526-61fa-4c62-b59b-cfd40b615713 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:45.508238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:02:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:02:45.513369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832b1db-ac59-47ae-9d20-9d246c7562aa None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:45.513777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832b1db-ac59-47ae-9d20-9d246c7562aa None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:45.513929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832b1db-ac59-47ae-9d20-9d246c7562aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:45.531645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e832b1db-ac59-47ae-9d20-9d246c7562aa None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:02:45.532830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e832b1db-ac59-47ae-9d20-9d246c7562aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:45.540831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:02:45.808686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dc526-61fa-4c62-b59b-cfd40b615713 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:02:45.810359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-400dc526-61fa-4c62-b59b-cfd40b615713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:02:45.823750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400dc526-61fa-4c62-b59b-cfd40b615713 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:02:45.827263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a557b64-e461-4879-89f5-fc4e34d786a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:02:45.827897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a557b64-e461-4879-89f5-fc4e34d786a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:02:45.828180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a557b64-e461-4879-89f5-fc4e34d786a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:02:45.828459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a557b64-e461-4879-89f5-fc4e34d786a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:02:45.837937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a557b64-e461-4879-89f5-fc4e34d786a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:02:45.839403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a557b64-e461-4879-89f5-fc4e34d786a4 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:02:45.848643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 767/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:02:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:02:55.526453 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:02:55.844514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:15.656490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e28985-2d7e-4bf0-b3e6-e6a9e51367f9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:15.657120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e28985-2d7e-4bf0-b3e6-e6a9e51367f9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:15.657120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e28985-2d7e-4bf0-b3e6-e6a9e51367f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:15.685147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-98e28985-2d7e-4bf0-b3e6-e6a9e51367f9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:15.686741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-98e28985-2d7e-4bf0-b3e6-e6a9e51367f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:15.702863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 773/1540] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:04:15.714415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795c97-ebcd-4a2b-8458-7cf0cb01389b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:15.715040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795c97-ebcd-4a2b-8458-7cf0cb01389b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:15.715040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c795c97-ebcd-4a2b-8458-7cf0cb01389b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:15.743408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4c795c97-ebcd-4a2b-8458-7cf0cb01389b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:15.744664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4c795c97-ebcd-4a2b-8458-7cf0cb01389b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:15.758004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 768/1541] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:15.763363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e28985-2d7e-4bf0-b3e6-e6a9e51367f9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:15.776721 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e28985-2d7e-4bf0-b3e6-e6a9e51367f9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:15.781678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c2db-1a6b-4932-8cbd-9cc3e2ff5a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:15.787648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c2db-1a6b-4932-8cbd-9cc3e2ff5a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:15.788263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c2db-1a6b-4932-8cbd-9cc3e2ff5a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:15.788777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643c2db-1a6b-4932-8cbd-9cc3e2ff5a28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:15.789795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c2db-1a6b-4932-8cbd-9cc3e2ff5a28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:15.790324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c2db-1a6b-4932-8cbd-9cc3e2ff5a28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:15.792188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643c2db-1a6b-4932-8cbd-9cc3e2ff5a28 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:15.812788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 774/1542] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:15.822568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c795c97-ebcd-4a2b-8458-7cf0cb01389b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:15.836125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c795c97-ebcd-4a2b-8458-7cf0cb01389b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:15.842732 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d30c7-3f89-4cb9-b728-315d4c30860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:15.843399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d30c7-3f89-4cb9-b728-315d4c30860a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:15.843684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d30c7-3f89-4cb9-b728-315d4c30860a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:15.843966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d30c7-3f89-4cb9-b728-315d4c30860a admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:15.844658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d30c7-3f89-4cb9-b728-315d4c30860a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:15.847283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d30c7-3f89-4cb9-b728-315d4c30860a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:15.897782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 769/1543] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:15 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:15.902616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c2db-1a6b-4932-8cbd-9cc3e2ff5a28 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:15.914318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643c2db-1a6b-4932-8cbd-9cc3e2ff5a28 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.026500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03b4f1-789f-454b-b33f-e33ec879506c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.026870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03b4f1-789f-454b-b33f-e33ec879506c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.027139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03b4f1-789f-454b-b33f-e33ec879506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.027555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd93acd-ab20-4bf7-a249-6819e33a4d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.028466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd93acd-ab20-4bf7-a249-6819e33a4d4c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.028959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd93acd-ab20-4bf7-a249-6819e33a4d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.029446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd93acd-ab20-4bf7-a249-6819e33a4d4c admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.030356 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd93acd-ab20-4bf7-a249-6819e33a4d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.032650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd93acd-ab20-4bf7-a249-6819e33a4d4c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.074899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5c03b4f1-789f-454b-b33f-e33ec879506c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.076432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5c03b4f1-789f-454b-b33f-e33ec879506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.086841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 775/1544] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:15 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:16.090711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd93acd-ab20-4bf7-a249-6819e33a4d4c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.091742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 770/1545] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:04:16.102472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921906c1-708b-4a28-a4ee-432761bb4813 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.102770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921906c1-708b-4a28-a4ee-432761bb4813 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.103032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921906c1-708b-4a28-a4ee-432761bb4813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.107687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd93acd-ab20-4bf7-a249-6819e33a4d4c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.113857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e42a22-ab79-4615-8457-afc3f974ee22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.115099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e42a22-ab79-4615-8457-afc3f974ee22 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.115765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e42a22-ab79-4615-8457-afc3f974ee22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.116253 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e42a22-ab79-4615-8457-afc3f974ee22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.117242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e42a22-ab79-4615-8457-afc3f974ee22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.120554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e42a22-ab79-4615-8457-afc3f974ee22 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.139904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 776/1546] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:04:16.141726 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-921906c1-708b-4a28-a4ee-432761bb4813 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.142474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e42a22-ab79-4615-8457-afc3f974ee22 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.142981 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-921906c1-708b-4a28-a4ee-432761bb4813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.154558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e42a22-ab79-4615-8457-afc3f974ee22 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.159892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 771/1547] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:04:16.161132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921906c1-708b-4a28-a4ee-432761bb4813 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.164313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733991c5-3f38-4499-80f2-35239670dad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.165894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733991c5-3f38-4499-80f2-35239670dad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.166426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733991c5-3f38-4499-80f2-35239670dad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.166944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733991c5-3f38-4499-80f2-35239670dad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.167901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733991c5-3f38-4499-80f2-35239670dad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.169812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733991c5-3f38-4499-80f2-35239670dad8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.188937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921906c1-708b-4a28-a4ee-432761bb4813 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.196354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 777/1548] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:04:16.201073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733991c5-3f38-4499-80f2-35239670dad8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.204216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db7b87-507a-481a-81fe-af1dc2e9bf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.204906 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db7b87-507a-481a-81fe-af1dc2e9bf49 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.205198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db7b87-507a-481a-81fe-af1dc2e9bf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.205479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db7b87-507a-481a-81fe-af1dc2e9bf49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.206268 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db7b87-507a-481a-81fe-af1dc2e9bf49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:16.206611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db7b87-507a-481a-81fe-af1dc2e9bf49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.207817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-a2db7b87-507a-481a-81fe-af1dc2e9bf49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 21:04:16.207817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 26 21:04:16.207817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: May 26 21:04:16.208874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db7b87-507a-481a-81fe-af1dc2e9bf49 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.220795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733991c5-3f38-4499-80f2-35239670dad8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.225927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941f71a-86c6-41c8-8a76-efcf79d82bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.227733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f71a-86c6-41c8-8a76-efcf79d82bab admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.228250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f71a-86c6-41c8-8a76-efcf79d82bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.228771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941f71a-86c6-41c8-8a76-efcf79d82bab admin admin] PATH_INFO: `/v3/projects/dab98c8a7d9e43249b01602eb3b63bb8/users/d1e6e50416af41d1af7d18cc5fb90253/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.237931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 772/1549] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:16.249275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db7b87-507a-481a-81fe-af1dc2e9bf49 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.256413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941f71a-86c6-41c8-8a76-efcf79d82bab admin admin] RBAC: Authorizing `identity:create_grant(project_id=dab98c8a7d9e43249b01602eb3b63bb8, user_id=d1e6e50416af41d1af7d18cc5fb90253, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.257810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-b941f71a-86c6-41c8-8a76-efcf79d82bab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:04:16.257810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: warnings.warn(msg) May 26 21:04:16.257810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: May 26 21:04:16.261280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941f71a-86c6-41c8-8a76-efcf79d82bab admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.269900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db7b87-507a-481a-81fe-af1dc2e9bf49 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.282748 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090217c5-4794-4d9f-859f-af1b97d0074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.283686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090217c5-4794-4d9f-859f-af1b97d0074d admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.284150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090217c5-4794-4d9f-859f-af1b97d0074d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.284548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090217c5-4794-4d9f-859f-af1b97d0074d admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.285641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090217c5-4794-4d9f-859f-af1b97d0074d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.291113 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090217c5-4794-4d9f-859f-af1b97d0074d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.311647 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 778/1550] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:16 2022] PUT /identity/v3/projects/dab98c8a7d9e43249b01602eb3b63bb8/users/d1e6e50416af41d1af7d18cc5fb90253/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:16.318259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bcc00-13aa-4f87-8f55-f1bac7079a99 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.318952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bcc00-13aa-4f87-8f55-f1bac7079a99 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.319472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bcc00-13aa-4f87-8f55-f1bac7079a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.351674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ac6bcc00-13aa-4f87-8f55-f1bac7079a99 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.354746 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ac6bcc00-13aa-4f87-8f55-f1bac7079a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.359066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 773/1551] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:16.364912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090217c5-4794-4d9f-859f-af1b97d0074d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.497673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090217c5-4794-4d9f-859f-af1b97d0074d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.499068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 779/1552] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.500944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19eaad7-5b25-4416-9e59-f3ddd95ff3ca None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.501458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19eaad7-5b25-4416-9e59-f3ddd95ff3ca None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.501955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19eaad7-5b25-4416-9e59-f3ddd95ff3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.514966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e2c48-46a6-412d-98a0-c61d62084bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.515671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e2c48-46a6-412d-98a0-c61d62084bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.515959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e2c48-46a6-412d-98a0-c61d62084bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.516240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e2c48-46a6-412d-98a0-c61d62084bd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.516949 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e2c48-46a6-412d-98a0-c61d62084bd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.517991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-889e2c48-46a6-412d-98a0-c61d62084bd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:04:16.517991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 26 21:04:16.517991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: May 26 21:04:16.519308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e2c48-46a6-412d-98a0-c61d62084bd4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.545352 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f19eaad7-5b25-4416-9e59-f3ddd95ff3ca None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.546666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f19eaad7-5b25-4416-9e59-f3ddd95ff3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.556005 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 780/1554] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.556463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 774/1554] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:04:16.557367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2eb3c-fc96-420f-82ca-63dd5ac463ae None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.557655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2eb3c-fc96-420f-82ca-63dd5ac463ae None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.557936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2eb3c-fc96-420f-82ca-63dd5ac463ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.558746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f211-c6da-4b1c-98c7-228c53405032 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.559410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f211-c6da-4b1c-98c7-228c53405032 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.559901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0f211-c6da-4b1c-98c7-228c53405032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.581677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-87c0f211-c6da-4b1c-98c7-228c53405032 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.583557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-87c0f211-c6da-4b1c-98c7-228c53405032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.585505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cfb2eb3c-fc96-420f-82ca-63dd5ac463ae None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.586711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cfb2eb3c-fc96-420f-82ca-63dd5ac463ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.592643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 775/1555] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.595076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 781/1556] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.596216 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd0193-4bbd-4d0a-95f5-8642011deb89 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.597031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd0193-4bbd-4d0a-95f5-8642011deb89 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.597308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd0193-4bbd-4d0a-95f5-8642011deb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.599063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0f211-c6da-4b1c-98c7-228c53405032 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.618234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cabd0193-4bbd-4d0a-95f5-8642011deb89 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.618492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cabd0193-4bbd-4d0a-95f5-8642011deb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.628286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 782/1557] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.629024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd0193-4bbd-4d0a-95f5-8642011deb89 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.635385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:16.639639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd0193-4bbd-4d0a-95f5-8642011deb89 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.646592 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739d98a-ab4f-4899-84c4-12ca2a2d25df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.647337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739d98a-ab4f-4899-84c4-12ca2a2d25df admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.647631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739d98a-ab4f-4899-84c4-12ca2a2d25df admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.647912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739d98a-ab4f-4899-84c4-12ca2a2d25df admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.648639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739d98a-ab4f-4899-84c4-12ca2a2d25df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.650241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739d98a-ab4f-4899-84c4-12ca2a2d25df admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.663910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 783/1558] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:16.663910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e754ce-6763-410b-a0b4-f116c402665d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.663910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e754ce-6763-410b-a0b4-f116c402665d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.663910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e754ce-6763-410b-a0b4-f116c402665d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.669674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0f211-c6da-4b1c-98c7-228c53405032 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.673369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb78dc-c288-4d75-a47a-763fc605d5f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.674009 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb78dc-c288-4d75-a47a-763fc605d5f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.674290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb78dc-c288-4d75-a47a-763fc605d5f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.675172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb78dc-c288-4d75-a47a-763fc605d5f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.680627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b4e754ce-6763-410b-a0b4-f116c402665d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.681821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b4e754ce-6763-410b-a0b4-f116c402665d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.684089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb78dc-c288-4d75-a47a-763fc605d5f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.685532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb78dc-c288-4d75-a47a-763fc605d5f8 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.689679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 784/1559] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.690924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daa41b-4b1e-4360-a7b0-272face6aa65 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.691211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daa41b-4b1e-4360-a7b0-272face6aa65 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.691488 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daa41b-4b1e-4360-a7b0-272face6aa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.698317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 776/1560] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:16.699656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72595f-416b-44f9-bc54-0ff867ae9543 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.699953 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72595f-416b-44f9-bc54-0ff867ae9543 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.700235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72595f-416b-44f9-bc54-0ff867ae9543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.709032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-86daa41b-4b1e-4360-a7b0-272face6aa65 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.710247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-86daa41b-4b1e-4360-a7b0-272face6aa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.719267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cb72595f-416b-44f9-bc54-0ff867ae9543 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.720458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cb72595f-416b-44f9-bc54-0ff867ae9543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.722905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 785/1561] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.724644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb28ec-1f25-4a7e-b531-ca6dd5db62be None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.725081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb28ec-1f25-4a7e-b531-ca6dd5db62be None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.725365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb28ec-1f25-4a7e-b531-ca6dd5db62be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.730534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 777/1562] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.731728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7ddfb-cd7b-4ea1-82ae-ea85d6d7f640 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.732014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7ddfb-cd7b-4ea1-82ae-ea85d6d7f640 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.732285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7ddfb-cd7b-4ea1-82ae-ea85d6d7f640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.739782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:16.753666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f1b7ddfb-cd7b-4ea1-82ae-ea85d6d7f640 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.754842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f1b7ddfb-cd7b-4ea1-82ae-ea85d6d7f640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.757919 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8aeb28ec-1f25-4a7e-b531-ca6dd5db62be None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.759208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8aeb28ec-1f25-4a7e-b531-ca6dd5db62be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.763100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 778/1563] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.764549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95552cfd-d840-4ac4-b68e-c45c05ff0312 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.764758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95552cfd-d840-4ac4-b68e-c45c05ff0312 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.765045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95552cfd-d840-4ac4-b68e-c45c05ff0312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.778599 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 786/1564] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.779432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb28ec-1f25-4a7e-b531-ca6dd5db62be None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.791618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb28ec-1f25-4a7e-b531-ca6dd5db62be None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.796433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-95552cfd-d840-4ac4-b68e-c45c05ff0312 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.797779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-95552cfd-d840-4ac4-b68e-c45c05ff0312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.798582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e762aa-5cf8-46fa-823b-30348b0e2e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.799313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e762aa-5cf8-46fa-823b-30348b0e2e52 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.800164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e762aa-5cf8-46fa-823b-30348b0e2e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.800445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e762aa-5cf8-46fa-823b-30348b0e2e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.802564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e762aa-5cf8-46fa-823b-30348b0e2e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.802937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e762aa-5cf8-46fa-823b-30348b0e2e52 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.809269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 779/1565] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.810523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95552cfd-d840-4ac4-b68e-c45c05ff0312 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.819071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 787/1566] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:16.820892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6892a56-dede-48c6-bb8c-83250fae0e68 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.821025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6892a56-dede-48c6-bb8c-83250fae0e68 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.821200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6892a56-dede-48c6-bb8c-83250fae0e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.825007 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95552cfd-d840-4ac4-b68e-c45c05ff0312 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.839080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d87be45-b912-47d2-833a-147e5f9d318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.840551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87be45-b912-47d2-833a-147e5f9d318f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.841287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87be45-b912-47d2-833a-147e5f9d318f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.841402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f6892a56-dede-48c6-bb8c-83250fae0e68 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.841904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d87be45-b912-47d2-833a-147e5f9d318f admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.842576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f6892a56-dede-48c6-bb8c-83250fae0e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.843441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87be45-b912-47d2-833a-147e5f9d318f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:16.844034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87be45-b912-47d2-833a-147e5f9d318f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.847100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d87be45-b912-47d2-833a-147e5f9d318f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.856471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 788/1567] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.856471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6892a56-dede-48c6-bb8c-83250fae0e68 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.870771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 780/1568] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:16.870967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6892a56-dede-48c6-bb8c-83250fae0e68 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.871070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341c01a-df1f-4f0f-8c2a-2dce891a5ed2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.871070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341c01a-df1f-4f0f-8c2a-2dce891a5ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.871070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341c01a-df1f-4f0f-8c2a-2dce891a5ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.875630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3119d-622b-43f5-a342-57e18bf32104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.876294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3119d-622b-43f5-a342-57e18bf32104 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.876579 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3119d-622b-43f5-a342-57e18bf32104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.877010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3119d-622b-43f5-a342-57e18bf32104 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.877580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3119d-622b-43f5-a342-57e18bf32104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:16.877921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3119d-622b-43f5-a342-57e18bf32104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.882328 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3119d-622b-43f5-a342-57e18bf32104 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.891330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d341c01a-df1f-4f0f-8c2a-2dce891a5ed2 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:16.892617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d341c01a-df1f-4f0f-8c2a-2dce891a5ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:16.902153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 789/1569] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:16.902153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3119d-622b-43f5-a342-57e18bf32104 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.913119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3119d-622b-43f5-a342-57e18bf32104 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.914636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 781/1570] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:16.915471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341c01a-df1f-4f0f-8c2a-2dce891a5ed2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.921934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1af1dc-c756-4791-ad6b-029df5858e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.922762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af1dc-c756-4791-ad6b-029df5858e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.923114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af1dc-c756-4791-ad6b-029df5858e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.923455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af1dc-c756-4791-ad6b-029df5858e25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.924272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1af1dc-c756-4791-ad6b-029df5858e25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:16.924701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1af1dc-c756-4791-ad6b-029df5858e25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.926043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341c01a-df1f-4f0f-8c2a-2dce891a5ed2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.926579 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1af1dc-c756-4791-ad6b-029df5858e25 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.931656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133150ab-70ad-4a14-903e-f89e59447473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.932580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133150ab-70ad-4a14-903e-f89e59447473 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.932853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133150ab-70ad-4a14-903e-f89e59447473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.933239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133150ab-70ad-4a14-903e-f89e59447473 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.934103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133150ab-70ad-4a14-903e-f89e59447473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:16.934494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133150ab-70ad-4a14-903e-f89e59447473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.936369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133150ab-70ad-4a14-903e-f89e59447473 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.950227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 790/1571] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:16.950227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1af1dc-c756-4791-ad6b-029df5858e25 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.960369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 782/1572] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:16.960369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133150ab-70ad-4a14-903e-f89e59447473 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:16.964677 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1af1dc-c756-4791-ad6b-029df5858e25 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.965115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133150ab-70ad-4a14-903e-f89e59447473 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:16.969984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041109d-f981-42a3-b1f0-040725101f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.970876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041109d-f981-42a3-b1f0-040725101f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.971128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041109d-f981-42a3-b1f0-040725101f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.972028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041109d-f981-42a3-b1f0-040725101f31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.972644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c029f-3c7e-40aa-975f-b2390975399f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:16.972773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041109d-f981-42a3-b1f0-040725101f31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:16.973093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041109d-f981-42a3-b1f0-040725101f31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.973573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c029f-3c7e-40aa-975f-b2390975399f admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:16.973864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c029f-3c7e-40aa-975f-b2390975399f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:16.974167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c029f-3c7e-40aa-975f-b2390975399f admin admin] PATH_INFO: `/v3/projects/aee8ddc72b8a4baeb01503afbd8aeab1/users/83f3cfb67d3147d7b856158dcf32ff03/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:16.974878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041109d-f981-42a3-b1f0-040725101f31 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.990123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c029f-3c7e-40aa-975f-b2390975399f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee8ddc72b8a4baeb01503afbd8aeab1, user_id=83f3cfb67d3147d7b856158dcf32ff03, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:16.999004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c029f-3c7e-40aa-975f-b2390975399f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:16.999166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 783/1573] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:16.999867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041109d-f981-42a3-b1f0-040725101f31 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.011634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 791/1574] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:16 2022] PUT /identity/v3/projects/aee8ddc72b8a4baeb01503afbd8aeab1/users/83f3cfb67d3147d7b856158dcf32ff03/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:17.011800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041109d-f981-42a3-b1f0-040725101f31 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.013241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c029f-3c7e-40aa-975f-b2390975399f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.085715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c029f-3c7e-40aa-975f-b2390975399f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.091652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3316ad-80d2-4848-938e-125314764c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.092524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3316ad-80d2-4848-938e-125314764c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.092893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3316ad-80d2-4848-938e-125314764c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.093189 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3316ad-80d2-4848-938e-125314764c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.094043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3316ad-80d2-4848-938e-125314764c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.094906 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9483e-0e79-435d-b26e-15e2d6b92615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.095237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-4f3316ad-80d2-4848-938e-125314764c79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:04:17.095237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: warnings.warn(msg) May 26 21:04:17.095237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: May 26 21:04:17.096281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9483e-0e79-435d-b26e-15e2d6b92615 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.097071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3316ad-80d2-4848-938e-125314764c79 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.097432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9483e-0e79-435d-b26e-15e2d6b92615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.098023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9483e-0e79-435d-b26e-15e2d6b92615 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.099796 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9483e-0e79-435d-b26e-15e2d6b92615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.102174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e9483e-0e79-435d-b26e-15e2d6b92615 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.127848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 792/1575] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.128603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3316ad-80d2-4848-938e-125314764c79 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.142940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 784/1576] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:16 2022] POST /identity/v3/projects => generated 389 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.145585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9483e-0e79-435d-b26e-15e2d6b92615 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.196813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:17.209202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3316ad-80d2-4848-938e-125314764c79 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.226067 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e9483e-0e79-435d-b26e-15e2d6b92615 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.231600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bcb3e-6fa0-4ada-9bce-a4adf912e684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.232268 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bcb3e-6fa0-4ada-9bce-a4adf912e684 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.232732 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bcb3e-6fa0-4ada-9bce-a4adf912e684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.233016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456bcb3e-6fa0-4ada-9bce-a4adf912e684 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.233723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bcb3e-6fa0-4ada-9bce-a4adf912e684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.235713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456bcb3e-6fa0-4ada-9bce-a4adf912e684 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.266861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 785/1577] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.268083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bcb3e-6fa0-4ada-9bce-a4adf912e684 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.277909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b711c3-a0e7-4d86-a5d1-ff7363f74b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.278679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b711c3-a0e7-4d86-a5d1-ff7363f74b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.278970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b711c3-a0e7-4d86-a5d1-ff7363f74b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.279250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b711c3-a0e7-4d86-a5d1-ff7363f74b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.279983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b711c3-a0e7-4d86-a5d1-ff7363f74b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:17.280295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b711c3-a0e7-4d86-a5d1-ff7363f74b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.282318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b711c3-a0e7-4d86-a5d1-ff7363f74b6d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.291017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:17.307293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 793/1578] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:17.308251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b711c3-a0e7-4d86-a5d1-ff7363f74b6d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.320072 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b711c3-a0e7-4d86-a5d1-ff7363f74b6d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.349982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456bcb3e-6fa0-4ada-9bce-a4adf912e684 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.355216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477eddc7-8d99-4d1e-8ee2-abdbd41a4283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.355882 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eddc7-8d99-4d1e-8ee2-abdbd41a4283 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.356187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eddc7-8d99-4d1e-8ee2-abdbd41a4283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.356457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477eddc7-8d99-4d1e-8ee2-abdbd41a4283 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.357158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477eddc7-8d99-4d1e-8ee2-abdbd41a4283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.359136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477eddc7-8d99-4d1e-8ee2-abdbd41a4283 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.388255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5aa69-c73c-4ba1-b249-44cb6fb46c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.389034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5aa69-c73c-4ba1-b249-44cb6fb46c11 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.389383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5aa69-c73c-4ba1-b249-44cb6fb46c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.389729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5aa69-c73c-4ba1-b249-44cb6fb46c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.390555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5aa69-c73c-4ba1-b249-44cb6fb46c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.390916 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 786/1579] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/projects => generated 357 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.392581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5aa69-c73c-4ba1-b249-44cb6fb46c11 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.392813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00787e92-9617-4da3-91c4-6a89c2b630f6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.393012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00787e92-9617-4da3-91c4-6a89c2b630f6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.393917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00787e92-9617-4da3-91c4-6a89c2b630f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.414283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-00787e92-9617-4da3-91c4-6a89c2b630f6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:17.415623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-00787e92-9617-4da3-91c4-6a89c2b630f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:17.422142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 794/1580] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.423178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5aa69-c73c-4ba1-b249-44cb6fb46c11 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.496358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5aa69-c73c-4ba1-b249-44cb6fb46c11 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.504196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 787/1581] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:17.504196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00787e92-9617-4da3-91c4-6a89c2b630f6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.507125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc38d7b-d747-4c4e-968a-a914a553c772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.508721 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38d7b-d747-4c4e-968a-a914a553c772 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.509299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38d7b-d747-4c4e-968a-a914a553c772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.510011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38d7b-d747-4c4e-968a-a914a553c772 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.510943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc38d7b-d747-4c4e-968a-a914a553c772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.512948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc38d7b-d747-4c4e-968a-a914a553c772 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.514269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00787e92-9617-4da3-91c4-6a89c2b630f6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.521040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9590431-f374-4d88-b972-c3fcc469c2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.522660 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9590431-f374-4d88-b972-c3fcc469c2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.522973 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9590431-f374-4d88-b972-c3fcc469c2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.523252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9590431-f374-4d88-b972-c3fcc469c2f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.523946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9590431-f374-4d88-b972-c3fcc469c2f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.526688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9590431-f374-4d88-b972-c3fcc469c2f6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.538960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 795/1582] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.539881 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc38d7b-d747-4c4e-968a-a914a553c772 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.548749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 788/1583] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/users => generated 295 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.549932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9590431-f374-4d88-b972-c3fcc469c2f6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.555130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc38d7b-d747-4c4e-968a-a914a553c772 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.559794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c99c-2ef2-40df-a72f-3901c85297db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.560466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c99c-2ef2-40df-a72f-3901c85297db admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.560753 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c99c-2ef2-40df-a72f-3901c85297db admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.561034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1c99c-2ef2-40df-a72f-3901c85297db admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.561287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9590431-f374-4d88-b972-c3fcc469c2f6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.561758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1c99c-2ef2-40df-a72f-3901c85297db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.563719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1c99c-2ef2-40df-a72f-3901c85297db admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.567150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc374-65df-4d28-b18b-bdc866c02100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.568171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc374-65df-4d28-b18b-bdc866c02100 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.570023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc374-65df-4d28-b18b-bdc866c02100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.570385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addc374-65df-4d28-b18b-bdc866c02100 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.571309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addc374-65df-4d28-b18b-bdc866c02100 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.571409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:17.574202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addc374-65df-4d28-b18b-bdc866c02100 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.592465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 796/1584] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.592465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c99c-2ef2-40df-a72f-3901c85297db admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.601736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1c99c-2ef2-40df-a72f-3901c85297db admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.606831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996ac49-c90b-464b-b635-6ec663b4d8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.607560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996ac49-c90b-464b-b635-6ec663b4d8cc admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.607846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996ac49-c90b-464b-b635-6ec663b4d8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.608173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996ac49-c90b-464b-b635-6ec663b4d8cc admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.608865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996ac49-c90b-464b-b635-6ec663b4d8cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.614282 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 789/1585] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.614425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996ac49-c90b-464b-b635-6ec663b4d8cc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.614566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc374-65df-4d28-b18b-bdc866c02100 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.642665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 797/1586] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/users => generated 279 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.647344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ff791-c873-40be-92bd-e8cc91c03ec0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.647654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ff791-c873-40be-92bd-e8cc91c03ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.647944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ff791-c873-40be-92bd-e8cc91c03ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.678606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-366ff791-c873-40be-92bd-e8cc91c03ec0 None None] MFA Rules not processed for user `d1e6e50416af41d1af7d18cc5fb90253`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:17.679829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-366ff791-c873-40be-92bd-e8cc91c03ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:17.691215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addc374-65df-4d28-b18b-bdc866c02100 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.695857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95f438-199a-4da7-a31f-7549cde0d7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.696519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95f438-199a-4da7-a31f-7549cde0d7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.696802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95f438-199a-4da7-a31f-7549cde0d7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.697224 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95f438-199a-4da7-a31f-7549cde0d7b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.697907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95f438-199a-4da7-a31f-7549cde0d7b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.699862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95f438-199a-4da7-a31f-7549cde0d7b6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.706489 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:17.721024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 790/1587] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.721944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95f438-199a-4da7-a31f-7549cde0d7b6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.723524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cb95f438-199a-4da7-a31f-7549cde0d7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:17.748214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 798/1588] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:17.749051 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ff791-c873-40be-92bd-e8cc91c03ec0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.759866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ff791-c873-40be-92bd-e8cc91c03ec0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.764378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895db168-a4a5-4c1e-9307-9afbd1875f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.765043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895db168-a4a5-4c1e-9307-9afbd1875f16 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.765337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895db168-a4a5-4c1e-9307-9afbd1875f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.765617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895db168-a4a5-4c1e-9307-9afbd1875f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.766342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895db168-a4a5-4c1e-9307-9afbd1875f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.767979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895db168-a4a5-4c1e-9307-9afbd1875f16 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.778253 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 799/1589] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:17.779141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895db168-a4a5-4c1e-9307-9afbd1875f16 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.788524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95f438-199a-4da7-a31f-7549cde0d7b6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.789041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895db168-a4a5-4c1e-9307-9afbd1875f16 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.792902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e168a05-3733-4c1d-94ba-0a000b9152e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.793545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e168a05-3733-4c1d-94ba-0a000b9152e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.793827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e168a05-3733-4c1d-94ba-0a000b9152e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.793931 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f53d4-949b-4854-8ed6-5a665b67df7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.794097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e168a05-3733-4c1d-94ba-0a000b9152e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.794544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f53d4-949b-4854-8ed6-5a665b67df7f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.794833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f53d4-949b-4854-8ed6-5a665b67df7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.795116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f53d4-949b-4854-8ed6-5a665b67df7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.795828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f53d4-949b-4854-8ed6-5a665b67df7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.797403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f53d4-949b-4854-8ed6-5a665b67df7f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.804150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e168a05-3733-4c1d-94ba-0a000b9152e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.805611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e168a05-3733-4c1d-94ba-0a000b9152e8 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.812185 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 800/1590] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:17.813043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f53d4-949b-4854-8ed6-5a665b67df7f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.818852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 791/1591] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:17.819707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e168a05-3733-4c1d-94ba-0a000b9152e8 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.823780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f53d4-949b-4854-8ed6-5a665b67df7f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.828624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a82ee-5f01-4cd8-8d9c-787c18a7c928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.829020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e168a05-3733-4c1d-94ba-0a000b9152e8 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.829280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a82ee-5f01-4cd8-8d9c-787c18a7c928 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.829686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a82ee-5f01-4cd8-8d9c-787c18a7c928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.829976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a82ee-5f01-4cd8-8d9c-787c18a7c928 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.830915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a82ee-5f01-4cd8-8d9c-787c18a7c928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.832648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a82ee-5f01-4cd8-8d9c-787c18a7c928 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.834177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aad6c-89e4-4614-9bb4-33cd4c53276e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.834890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aad6c-89e4-4614-9bb4-33cd4c53276e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.835221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aad6c-89e4-4614-9bb4-33cd4c53276e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.835501 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454aad6c-89e4-4614-9bb4-33cd4c53276e admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.836187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aad6c-89e4-4614-9bb4-33cd4c53276e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.838310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454aad6c-89e4-4614-9bb4-33cd4c53276e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.845516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 801/1592] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:17.848566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a82ee-5f01-4cd8-8d9c-787c18a7c928 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.858019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a82ee-5f01-4cd8-8d9c-787c18a7c928 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.866351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 792/1593] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:17.866635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa760617-9a94-4e91-9bdc-e5335302bd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.866764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aad6c-89e4-4614-9bb4-33cd4c53276e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.866865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa760617-9a94-4e91-9bdc-e5335302bd2b admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.866865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa760617-9a94-4e91-9bdc-e5335302bd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.866865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa760617-9a94-4e91-9bdc-e5335302bd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.866865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa760617-9a94-4e91-9bdc-e5335302bd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.867502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa760617-9a94-4e91-9bdc-e5335302bd2b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.878142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454aad6c-89e4-4614-9bb4-33cd4c53276e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.883794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 802/1594] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:17.883794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e114ea0-2bd6-4fe4-9f83-07366c13b98a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.883794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e114ea0-2bd6-4fe4-9f83-07366c13b98a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.883794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e114ea0-2bd6-4fe4-9f83-07366c13b98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.887134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd8a01-2625-4eb7-aea7-081370cf1400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.887970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd8a01-2625-4eb7-aea7-081370cf1400 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.888514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd8a01-2625-4eb7-aea7-081370cf1400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.888831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd8a01-2625-4eb7-aea7-081370cf1400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.889793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd8a01-2625-4eb7-aea7-081370cf1400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.891389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd8a01-2625-4eb7-aea7-081370cf1400 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.905323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9e114ea0-2bd6-4fe4-9f83-07366c13b98a None None] MFA Rules not processed for user `d1e6e50416af41d1af7d18cc5fb90253`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:17.908571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 793/1595] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:17.908571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd8a01-2625-4eb7-aea7-081370cf1400 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.908873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e114ea0-2bd6-4fe4-9f83-07366c13b98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:17.916484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 803/1596] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:17.919755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd8a01-2625-4eb7-aea7-081370cf1400 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.921738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e114ea0-2bd6-4fe4-9f83-07366c13b98a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.926673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00201ff4-1b4f-486b-af18-96f797192e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.927355 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00201ff4-1b4f-486b-af18-96f797192e99 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.927908 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00201ff4-1b4f-486b-af18-96f797192e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.928128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00201ff4-1b4f-486b-af18-96f797192e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.929071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00201ff4-1b4f-486b-af18-96f797192e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.931309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00201ff4-1b4f-486b-af18-96f797192e99 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.933924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e114ea0-2bd6-4fe4-9f83-07366c13b98a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.939633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9c634-2497-48e4-9141-840c2aec4b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.942133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9c634-2497-48e4-9141-840c2aec4b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.942650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9c634-2497-48e4-9141-840c2aec4b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.943130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9c634-2497-48e4-9141-840c2aec4b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.944233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9c634-2497-48e4-9141-840c2aec4b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.948186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 794/1597] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:17.948186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00201ff4-1b4f-486b-af18-96f797192e99 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.950208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9c634-2497-48e4-9141-840c2aec4b2c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.962574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00201ff4-1b4f-486b-af18-96f797192e99 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.964740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 804/1598] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:17.966033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9c634-2497-48e4-9141-840c2aec4b2c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.969603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a8248-4bca-456f-b55e-b229ab54fd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.970313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a8248-4bca-456f-b55e-b229ab54fd3d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.970643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a8248-4bca-456f-b55e-b229ab54fd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.970952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a8248-4bca-456f-b55e-b229ab54fd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.972415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a8248-4bca-456f-b55e-b229ab54fd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.974050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a8248-4bca-456f-b55e-b229ab54fd3d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.978714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9c634-2497-48e4-9141-840c2aec4b2c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.983407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05dcd56-22a2-4cad-a986-cb0102573197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:17.984242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05dcd56-22a2-4cad-a986-cb0102573197 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:17.984526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05dcd56-22a2-4cad-a986-cb0102573197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:17.984805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05dcd56-22a2-4cad-a986-cb0102573197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:17.985534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05dcd56-22a2-4cad-a986-cb0102573197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:17.985670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 795/1599] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:17.986384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a8248-4bca-456f-b55e-b229ab54fd3d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:17.987730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05dcd56-22a2-4cad-a986-cb0102573197 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:17.999159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a8248-4bca-456f-b55e-b229ab54fd3d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:17.999345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 805/1600] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:18.000343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05dcd56-22a2-4cad-a986-cb0102573197 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:18.005890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e979cc-f529-404c-ae91-984540d418d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:18.006613 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e979cc-f529-404c-ae91-984540d418d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.006903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e979cc-f529-404c-ae91-984540d418d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.007191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e979cc-f529-404c-ae91-984540d418d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.008559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e979cc-f529-404c-ae91-984540d418d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:18.009552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e979cc-f529-404c-ae91-984540d418d2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:18.017105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05dcd56-22a2-4cad-a986-cb0102573197 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:18.022169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 796/1601] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:18.024008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ee9a0-6d99-4bdf-9a3a-b0cf5eec4271 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.024311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ee9a0-6d99-4bdf-9a3a-b0cf5eec4271 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.024600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ee9a0-6d99-4bdf-9a3a-b0cf5eec4271 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.026568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 797/1602] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 26 21:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:04:18.027393 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ee9a0-6d99-4bdf-9a3a-b0cf5eec4271 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:18.034514 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847411f-d887-460b-a877-31c1dbfaac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:18.035349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847411f-d887-460b-a877-31c1dbfaac0c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.035649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847411f-d887-460b-a877-31c1dbfaac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.035934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847411f-d887-460b-a877-31c1dbfaac0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.036721 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847411f-d887-460b-a877-31c1dbfaac0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:18.038354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847411f-d887-460b-a877-31c1dbfaac0c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:18.042195 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ee9a0-6d99-4bdf-9a3a-b0cf5eec4271 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:18.047975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f8c4e-afb1-4042-a21e-fb9df78811e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:18.048943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f8c4e-afb1-4042-a21e-fb9df78811e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.049301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f8c4e-afb1-4042-a21e-fb9df78811e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.049692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f8c4e-afb1-4042-a21e-fb9df78811e0 admin admin] PATH_INFO: `/v3/projects/e3fdd66858554009b0ccd42d6b8d11f5/users/1168b6a97b30483f8e3aed805200d7c5/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.053828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 806/1603] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:18.054207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847411f-d887-460b-a877-31c1dbfaac0c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:18.063732 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847411f-d887-460b-a877-31c1dbfaac0c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:18.067856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f8c4e-afb1-4042-a21e-fb9df78811e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fdd66858554009b0ccd42d6b8d11f5, user_id=1168b6a97b30483f8e3aed805200d7c5, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:18.069319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6619ae-017a-46fa-aa09-ddec44c4c335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:18.070301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6619ae-017a-46fa-aa09-ddec44c4c335 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.070695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6619ae-017a-46fa-aa09-ddec44c4c335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.070811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f8c4e-afb1-4042-a21e-fb9df78811e0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:18.071000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6619ae-017a-46fa-aa09-ddec44c4c335 admin admin] PATH_INFO: `/v3/projects/ba383df93dcf4c32b2b1c17bebd7f752/users/8145168b1d02438886cd0865844f7480/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.087592 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6619ae-017a-46fa-aa09-ddec44c4c335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba383df93dcf4c32b2b1c17bebd7f752, user_id=8145168b1d02438886cd0865844f7480, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:18.088267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 798/1604] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:18 2022] PUT /identity/v3/projects/e3fdd66858554009b0ccd42d6b8d11f5/users/1168b6a97b30483f8e3aed805200d7c5/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:18.088935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f8c4e-afb1-4042-a21e-fb9df78811e0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:18.097484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6619ae-017a-46fa-aa09-ddec44c4c335 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:18.114125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 807/1605] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:18 2022] PUT /identity/v3/projects/ba383df93dcf4c32b2b1c17bebd7f752/users/8145168b1d02438886cd0865844f7480/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:18.115248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6619ae-017a-46fa-aa09-ddec44c4c335 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:18.130604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:18.163997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f8c4e-afb1-4042-a21e-fb9df78811e0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:18.178151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:18.189332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6619ae-017a-46fa-aa09-ddec44c4c335 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:18.196361 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5dbcd-b245-40dd-9dad-b2ff6ee02ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:18.197877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5dbcd-b245-40dd-9dad-b2ff6ee02ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.198628 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5dbcd-b245-40dd-9dad-b2ff6ee02ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.198917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da5dbcd-b245-40dd-9dad-b2ff6ee02ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.199999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da5dbcd-b245-40dd-9dad-b2ff6ee02ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:18.202819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da5dbcd-b245-40dd-9dad-b2ff6ee02ad6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:18.216763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 808/1606] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:18.218385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5dbcd-b245-40dd-9dad-b2ff6ee02ad6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:18.231859 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da5dbcd-b245-40dd-9dad-b2ff6ee02ad6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:18.238534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c3f75-6511-4e8c-8c76-f3e9164a8ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:18.239600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c3f75-6511-4e8c-8c76-f3e9164a8ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.240006 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c3f75-6511-4e8c-8c76-f3e9164a8ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.240472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c3f75-6511-4e8c-8c76-f3e9164a8ed8 admin admin] PATH_INFO: `/v3/projects/73887c3d2f5a4a2793387257395c44d8/users/dd0d57d7bf834ef98b450530f3b3c70f/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.241956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051fa5d0-f948-4198-8511-5a23f5a5655a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:18.242989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fa5d0-f948-4198-8511-5a23f5a5655a admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.243377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fa5d0-f948-4198-8511-5a23f5a5655a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.243755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051fa5d0-f948-4198-8511-5a23f5a5655a admin admin] PATH_INFO: `/v3/projects/2aeed4619d4f4528ba73300df3d7a07d/users/a0c0eb6b15754e71908441206e7be36c/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.262523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fa5d0-f948-4198-8511-5a23f5a5655a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aeed4619d4f4528ba73300df3d7a07d, user_id=a0c0eb6b15754e71908441206e7be36c, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:18.265776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051fa5d0-f948-4198-8511-5a23f5a5655a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:18.275414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c3f75-6511-4e8c-8c76-f3e9164a8ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73887c3d2f5a4a2793387257395c44d8, user_id=dd0d57d7bf834ef98b450530f3b3c70f, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:18.278278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c3f75-6511-4e8c-8c76-f3e9164a8ed8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:18.288309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 799/1607] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:18 2022] PUT /identity/v3/projects/2aeed4619d4f4528ba73300df3d7a07d/users/a0c0eb6b15754e71908441206e7be36c/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:18.289947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a784a2f-129e-4a24-be94-7ccde0a00510 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.290395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a784a2f-129e-4a24-be94-7ccde0a00510 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.290705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a784a2f-129e-4a24-be94-7ccde0a00510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.305358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 809/1608] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:18 2022] PUT /identity/v3/projects/73887c3d2f5a4a2793387257395c44d8/users/dd0d57d7bf834ef98b450530f3b3c70f/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:18.306736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c3f75-6511-4e8c-8c76-f3e9164a8ed8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:18.310948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3a784a2f-129e-4a24-be94-7ccde0a00510 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:18.312290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3a784a2f-129e-4a24-be94-7ccde0a00510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:18.392359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c3f75-6511-4e8c-8c76-f3e9164a8ed8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:18.396664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 800/1609] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 26 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:18.398225 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fe313-38d5-48c6-8ad3-1ac3dfc7d944 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.398536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fe313-38d5-48c6-8ad3-1ac3dfc7d944 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.398819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fe313-38d5-48c6-8ad3-1ac3dfc7d944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.399200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066ac0c-6338-4496-bf82-33f25a64d7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:18.400649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066ac0c-6338-4496-bf82-33f25a64d7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.401395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066ac0c-6338-4496-bf82-33f25a64d7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.402000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066ac0c-6338-4496-bf82-33f25a64d7fd admin admin] PATH_INFO: `/v3/projects/8b4ba0955acb4f48845df8e538376807/users/324ff7734b0c42869259f8682847b80b/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.421876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066ac0c-6338-4496-bf82-33f25a64d7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4ba0955acb4f48845df8e538376807, user_id=324ff7734b0c42869259f8682847b80b, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:18.425235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066ac0c-6338-4496-bf82-33f25a64d7fd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:18.432818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-900fe313-38d5-48c6-8ad3-1ac3dfc7d944 None None] MFA Rules not processed for user `1168b6a97b30483f8e3aed805200d7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:18.434178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-900fe313-38d5-48c6-8ad3-1ac3dfc7d944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:18.445717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 810/1610] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:18 2022] PUT /identity/v3/projects/8b4ba0955acb4f48845df8e538376807/users/324ff7734b0c42869259f8682847b80b/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:18.447409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b963bb2-a42d-4ae0-9df2-120530a11614 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.447701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b963bb2-a42d-4ae0-9df2-120530a11614 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.448106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b963bb2-a42d-4ae0-9df2-120530a11614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.469034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0b963bb2-a42d-4ae0-9df2-120530a11614 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:18.470709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0b963bb2-a42d-4ae0-9df2-120530a11614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:18.515429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 801/1611] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:18.516980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fe313-38d5-48c6-8ad3-1ac3dfc7d944 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:18.530803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fe313-38d5-48c6-8ad3-1ac3dfc7d944 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:18.545232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:18.554294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 811/1612] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:18.562464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c79ef-8e06-4e04-b6bf-a059837cc93a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.564132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c79ef-8e06-4e04-b6bf-a059837cc93a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.564491 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c79ef-8e06-4e04-b6bf-a059837cc93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.592735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:18.602969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c5064-f9f2-4fb9-83f9-658f764a1ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:18.603875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c5064-f9f2-4fb9-83f9-658f764a1ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.604160 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c5064-f9f2-4fb9-83f9-658f764a1ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.604441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c5064-f9f2-4fb9-83f9-658f764a1ab2 admin admin] PATH_INFO: `/v3/projects/9f8ac962c5d0402eb17b01aa02632897/users/443be2bc27514432a34d2a4c57108b7a/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.607855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-428c79ef-8e06-4e04-b6bf-a059837cc93a None None] MFA Rules not processed for user `a0c0eb6b15754e71908441206e7be36c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:18.610856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-428c79ef-8e06-4e04-b6bf-a059837cc93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:18.633086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c5064-f9f2-4fb9-83f9-658f764a1ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8ac962c5d0402eb17b01aa02632897, user_id=443be2bc27514432a34d2a4c57108b7a, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:18.636249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c5064-f9f2-4fb9-83f9-658f764a1ab2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:18.657285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 802/1613] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:18 2022] PUT /identity/v3/projects/9f8ac962c5d0402eb17b01aa02632897/users/443be2bc27514432a34d2a4c57108b7a/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:18.658653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e7dd-027d-4050-af16-d3b3e31292e6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.659056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e7dd-027d-4050-af16-d3b3e31292e6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.659347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74e7dd-027d-4050-af16-d3b3e31292e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.695213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8c74e7dd-027d-4050-af16-d3b3e31292e6 None None] MFA Rules not processed for user `dd0d57d7bf834ef98b450530f3b3c70f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:18.696883 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8c74e7dd-027d-4050-af16-d3b3e31292e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:18.702602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 812/1614] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:18.704715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c79ef-8e06-4e04-b6bf-a059837cc93a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:18.808764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 803/1615] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:18.813904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c38ce-38e8-4287-9c29-da7f30f53829 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.814797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c38ce-38e8-4287-9c29-da7f30f53829 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.814797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c38ce-38e8-4287-9c29-da7f30f53829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.839489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c79ef-8e06-4e04-b6bf-a059837cc93a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:18.853620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-051c38ce-38e8-4287-9c29-da7f30f53829 None None] MFA Rules not processed for user `324ff7734b0c42869259f8682847b80b`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:18.855396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-051c38ce-38e8-4287-9c29-da7f30f53829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:18.921079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8755135-5b2e-4ba1-a8a8-86df2302b39f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:18.921961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8755135-5b2e-4ba1-a8a8-86df2302b39f service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:18.922226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8755135-5b2e-4ba1-a8a8-86df2302b39f service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:18.922674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8755135-5b2e-4ba1-a8a8-86df2302b39f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:18.942945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 804/1616] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:18.944500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c38ce-38e8-4287-9c29-da7f30f53829 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:18.967292 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:18.970914 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:19.007457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8755135-5b2e-4ba1-a8a8-86df2302b39f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.009452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8755135-5b2e-4ba1-a8a8-86df2302b39f service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:19.012873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c38ce-38e8-4287-9c29-da7f30f53829 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:19.017257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696b363-9bb0-4880-b411-89d836acd379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:19.018174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696b363-9bb0-4880-b411-89d836acd379 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.018741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696b363-9bb0-4880-b411-89d836acd379 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.019098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696b363-9bb0-4880-b411-89d836acd379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.021717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 813/1617] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:19.022988 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8755135-5b2e-4ba1-a8a8-86df2302b39f service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:19.090896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8755135-5b2e-4ba1-a8a8-86df2302b39f service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:19.095119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb00731-9e47-4a54-8c28-cde915cef680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:19.095833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb00731-9e47-4a54-8c28-cde915cef680 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.096127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb00731-9e47-4a54-8c28-cde915cef680 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.097203 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb00731-9e47-4a54-8c28-cde915cef680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.116563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696b363-9bb0-4880-b411-89d836acd379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.118570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696b363-9bb0-4880-b411-89d836acd379 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:19.129434 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 805/1618] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:19.131270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e3062-0f01-4f2f-b4e6-402af76e9b81 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.131899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e3062-0f01-4f2f-b4e6-402af76e9b81 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.132455 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e3062-0f01-4f2f-b4e6-402af76e9b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.157728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f32e3062-0f01-4f2f-b4e6-402af76e9b81 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:19.164747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f32e3062-0f01-4f2f-b4e6-402af76e9b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:19.200041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb00731-9e47-4a54-8c28-cde915cef680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.201609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb00731-9e47-4a54-8c28-cde915cef680 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:19.214534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 814/1619] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:19.216248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7483e-592b-42d2-8394-f00667909e2a None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.216548 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7483e-592b-42d2-8394-f00667909e2a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.216835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7483e-592b-42d2-8394-f00667909e2a None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.218414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 815/1620] 10.222.0.57 () {54 vars in 961 bytes} [Thu May 26 21:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:04:19.219421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7483e-592b-42d2-8394-f00667909e2a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:19.243464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 806/1621] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:19.246584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89086050-d638-499e-be04-e8e697c415d1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.247365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89086050-d638-499e-be04-e8e697c415d1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.247756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89086050-d638-499e-be04-e8e697c415d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.257474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7483e-592b-42d2-8394-f00667909e2a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:19.272115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-89086050-d638-499e-be04-e8e697c415d1 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:19.272377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8aa41-df7d-4dba-ada6-f201d5a0eb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:19.272943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8aa41-df7d-4dba-ada6-f201d5a0eb34 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.273233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8aa41-df7d-4dba-ada6-f201d5a0eb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.273803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8aa41-df7d-4dba-ada6-f201d5a0eb34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.273940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-89086050-d638-499e-be04-e8e697c415d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:19.274581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8aa41-df7d-4dba-ada6-f201d5a0eb34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.276894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8aa41-df7d-4dba-ada6-f201d5a0eb34 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:19.286383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 807/1622] 10.222.0.57 () {58 vars in 1058 bytes} [Thu May 26 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:19.287655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89086050-d638-499e-be04-e8e697c415d1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:19.300847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89086050-d638-499e-be04-e8e697c415d1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:19.305633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abead1d8-4776-4d81-b450-c34c062c7d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:19.306021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abead1d8-4776-4d81-b450-c34c062c7d67 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.306435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abead1d8-4776-4d81-b450-c34c062c7d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.307035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abead1d8-4776-4d81-b450-c34c062c7d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.321540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abead1d8-4776-4d81-b450-c34c062c7d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.323452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abead1d8-4776-4d81-b450-c34c062c7d67 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:19.340678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 808/1623] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:19.342283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abead1d8-4776-4d81-b450-c34c062c7d67 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:19.343953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 816/1624] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:19 2022] POST /identity/v3/projects => generated 355 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:19.348651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8aa41-df7d-4dba-ada6-f201d5a0eb34 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:19.372187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abead1d8-4776-4d81-b450-c34c062c7d67 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:19.453221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:19.473050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645a969-e734-45c6-90fc-96b91cd4d8b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:19.474528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645a969-e734-45c6-90fc-96b91cd4d8b3 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.475156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645a969-e734-45c6-90fc-96b91cd4d8b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.475319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645a969-e734-45c6-90fc-96b91cd4d8b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.483982 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8aa41-df7d-4dba-ada6-f201d5a0eb34 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:19.503299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612fab0-8e84-4c5f-92e8-9e9ec68d13c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:19.504085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612fab0-8e84-4c5f-92e8-9e9ec68d13c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.504375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612fab0-8e84-4c5f-92e8-9e9ec68d13c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.504653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612fab0-8e84-4c5f-92e8-9e9ec68d13c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.505451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612fab0-8e84-4c5f-92e8-9e9ec68d13c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.507686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612fab0-8e84-4c5f-92e8-9e9ec68d13c3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:19.522379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:19.551959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 817/1625] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:19 2022] POST /identity/v3/users => generated 284 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:04:19.555705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612fab0-8e84-4c5f-92e8-9e9ec68d13c3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:19.575344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612fab0-8e84-4c5f-92e8-9e9ec68d13c3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:19.588288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d86c9f-ecba-45d9-93c7-08e1e99285f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:19.591039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d86c9f-ecba-45d9-93c7-08e1e99285f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.591334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d86c9f-ecba-45d9-93c7-08e1e99285f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.591665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d86c9f-ecba-45d9-93c7-08e1e99285f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.593631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d86c9f-ecba-45d9-93c7-08e1e99285f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.597531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d86c9f-ecba-45d9-93c7-08e1e99285f3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:19.625645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 818/1626] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:04:19.638322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d86c9f-ecba-45d9-93c7-08e1e99285f3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:19.668053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645a969-e734-45c6-90fc-96b91cd4d8b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.670425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d86c9f-ecba-45d9-93c7-08e1e99285f3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:19.674880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645a969-e734-45c6-90fc-96b91cd4d8b3 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:19.685676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3767d-8c9c-45a3-955f-c048db74d948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:19.687665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3767d-8c9c-45a3-955f-c048db74d948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.688327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3767d-8c9c-45a3-955f-c048db74d948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.689082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3767d-8c9c-45a3-955f-c048db74d948 admin admin] PATH_INFO: `/v3/projects/a82ff6593d3e4905b70d250bc3dfc60c/users/7a7c2ae9eb17496fb038328f66e86ffe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.708385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 809/1627] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:19.713664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3767d-8c9c-45a3-955f-c048db74d948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82ff6593d3e4905b70d250bc3dfc60c, user_id=7a7c2ae9eb17496fb038328f66e86ffe, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.717936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3767d-8c9c-45a3-955f-c048db74d948 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:19.742926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 819/1628] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:19 2022] PUT /identity/v3/projects/a82ff6593d3e4905b70d250bc3dfc60c/users/7a7c2ae9eb17496fb038328f66e86ffe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:19.746852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645a969-e734-45c6-90fc-96b91cd4d8b3 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:19.763253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645a969-e734-45c6-90fc-96b91cd4d8b3 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:19.808272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3651d-6d29-4269-aec1-b49b873e5195 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.809104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3651d-6d29-4269-aec1-b49b873e5195 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.809911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3651d-6d29-4269-aec1-b49b873e5195 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.813094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 820/1629] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 26 21:04:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:04:19.820170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b090f70-8a15-4ebe-98f2-a85094f8a276 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.821034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b090f70-8a15-4ebe-98f2-a85094f8a276 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.821690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b090f70-8a15-4ebe-98f2-a85094f8a276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.859563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8b090f70-8a15-4ebe-98f2-a85094f8a276 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:19.861619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8b090f70-8a15-4ebe-98f2-a85094f8a276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:19.872256 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f7945-e7e9-4652-af9c-77459588171d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:19.875742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f7945-e7e9-4652-af9c-77459588171d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.878066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f7945-e7e9-4652-af9c-77459588171d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.880046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f7945-e7e9-4652-af9c-77459588171d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.882730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f7945-e7e9-4652-af9c-77459588171d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.889479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f7945-e7e9-4652-af9c-77459588171d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:19.919165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 810/1630] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:19.920853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f7945-e7e9-4652-af9c-77459588171d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:19.943877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f7945-e7e9-4652-af9c-77459588171d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:19.951062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ec417-d040-4020-b867-ce5703449fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:19.952486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ec417-d040-4020-b867-ce5703449fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:19.952873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ec417-d040-4020-b867-ce5703449fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:19.953275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ec417-d040-4020-b867-ce5703449fe8 admin admin] PATH_INFO: `/v3/projects/a82ff6593d3e4905b70d250bc3dfc60c/users/7a7c2ae9eb17496fb038328f66e86ffe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:19.956637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ec417-d040-4020-b867-ce5703449fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a82ff6593d3e4905b70d250bc3dfc60c, user_id=7a7c2ae9eb17496fb038328f66e86ffe, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:19.966037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ec417-d040-4020-b867-ce5703449fe8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:20.069113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 821/1631] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:20.088899 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b090f70-8a15-4ebe-98f2-a85094f8a276 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:20.092580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 811/1632] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:19 2022] PUT /identity/v3/projects/a82ff6593d3e4905b70d250bc3dfc60c/users/7a7c2ae9eb17496fb038328f66e86ffe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:20.094633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8b090f70-8a15-4ebe-98f2-a85094f8a276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:20.105528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fd753-dd9d-4302-bfb0-0ba70cd7a47f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:20.105817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fd753-dd9d-4302-bfb0-0ba70cd7a47f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:20.106102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fd753-dd9d-4302-bfb0-0ba70cd7a47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:20.157927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a75fd753-dd9d-4302-bfb0-0ba70cd7a47f None None] MFA Rules not processed for user `7a7c2ae9eb17496fb038328f66e86ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:20.159379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a75fd753-dd9d-4302-bfb0-0ba70cd7a47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:20.245946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:20.261027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 812/1633] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:20 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:20.262241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fd753-dd9d-4302-bfb0-0ba70cd7a47f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:20.302867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b090f70-8a15-4ebe-98f2-a85094f8a276 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:20.383146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fd753-dd9d-4302-bfb0-0ba70cd7a47f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:20.389766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f73aa-1d50-447d-aca3-ffb2fc5a8694 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3bd8dd462846bf9423f8528aa5c489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:20.390885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d7972-74b7-47d6-b68d-8ba4490e3de4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:20.391153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f73aa-1d50-447d-aca3-ffb2fc5a8694 service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:20.391632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f73aa-1d50-447d-aca3-ffb2fc5a8694 service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:20.392023 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f73aa-1d50-447d-aca3-ffb2fc5a8694 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:20.392644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d7972-74b7-47d6-b68d-8ba4490e3de4 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:20.393316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d7972-74b7-47d6-b68d-8ba4490e3de4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:20.394016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d7972-74b7-47d6-b68d-8ba4490e3de4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:20.415854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d7972-74b7-47d6-b68d-8ba4490e3de4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:20.419500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d7972-74b7-47d6-b68d-8ba4490e3de4 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:20.435246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 813/1634] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:20.437012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f13381-f508-4ab0-bf28-68312d2062ec None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:20.437480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f13381-f508-4ab0-bf28-68312d2062ec None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:20.437797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f13381-f508-4ab0-bf28-68312d2062ec None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:20.443741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 814/1635] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 26 21:04:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:04:20.444985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf1537-88d7-4a90-8124-3c4c6502eb06 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:20.445283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf1537-88d7-4a90-8124-3c4c6502eb06 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:20.445570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbf1537-88d7-4a90-8124-3c4c6502eb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:20.453591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:20.467536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-efbf1537-88d7-4a90-8124-3c4c6502eb06 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:20.469241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-efbf1537-88d7-4a90-8124-3c4c6502eb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:20.540602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f73aa-1d50-447d-aca3-ffb2fc5a8694 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:20.542738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f73aa-1d50-447d-aca3-ffb2fc5a8694 service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:20.578352 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 822/1636] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:20.579458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f73aa-1d50-447d-aca3-ffb2fc5a8694 service glance] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:20.592639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 815/1637] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:04:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:20.600339 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f73aa-1d50-447d-aca3-ffb2fc5a8694 service glance] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:20.606387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cddcd4-87e1-4a59-a725-10e8ce092f16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:20.607150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cddcd4-87e1-4a59-a725-10e8ce092f16 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:20.607514 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cddcd4-87e1-4a59-a725-10e8ce092f16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:20.607906 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cddcd4-87e1-4a59-a725-10e8ce092f16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:20.629366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf1537-88d7-4a90-8124-3c4c6502eb06 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:20.648210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbf1537-88d7-4a90-8124-3c4c6502eb06 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:20.653937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac724218-0062-4c09-99b7-6ad82a27f4c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:20.654766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac724218-0062-4c09-99b7-6ad82a27f4c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:20.655858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac724218-0062-4c09-99b7-6ad82a27f4c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:20.656153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac724218-0062-4c09-99b7-6ad82a27f4c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:20.672379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac724218-0062-4c09-99b7-6ad82a27f4c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:20.674535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac724218-0062-4c09-99b7-6ad82a27f4c2 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:20.689393 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cddcd4-87e1-4a59-a725-10e8ce092f16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:20.690972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cddcd4-87e1-4a59-a725-10e8ce092f16 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:20.699077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 816/1638] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:20.716679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 823/1639] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:21.123339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61710801-ccd9-4f70-b8a3-ea00ced16e97 None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:21.123879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61710801-ccd9-4f70-b8a3-ea00ced16e97 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:21.123879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61710801-ccd9-4f70-b8a3-ea00ced16e97 None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:21.125758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 817/1640] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 26 21:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:04:21.132804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989b9b-6751-491d-9768-f53407e4d697 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:21.133286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989b9b-6751-491d-9768-f53407e4d697 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:21.133665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989b9b-6751-491d-9768-f53407e4d697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:21.181277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9e989b9b-6751-491d-9768-f53407e4d697 None None] MFA Rules not processed for user `94b8d8b089264b5192a651b41bf2a205`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:21.182767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e989b9b-6751-491d-9768-f53407e4d697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:21.262195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 824/1641] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:04:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:21.360729 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61710801-ccd9-4f70-b8a3-ea00ced16e97 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:21.382832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61710801-ccd9-4f70-b8a3-ea00ced16e97 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:21.395249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d316e-e4ab-4c1f-ac35-5c0ec988aeff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:21.395921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d316e-e4ab-4c1f-ac35-5c0ec988aeff service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:21.396207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d316e-e4ab-4c1f-ac35-5c0ec988aeff service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:21.396485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d316e-e4ab-4c1f-ac35-5c0ec988aeff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:21.419472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d316e-e4ab-4c1f-ac35-5c0ec988aeff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:21.421122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d316e-e4ab-4c1f-ac35-5c0ec988aeff service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:21.438708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 818/1642] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:21.530156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e989b9b-6751-491d-9768-f53407e4d697 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:21.550486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e989b9b-6751-491d-9768-f53407e4d697 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:21.570233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a4e70-00f8-464b-a8e0-41e1c091a660 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:21.570996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a4e70-00f8-464b-a8e0-41e1c091a660 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:21.571289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a4e70-00f8-464b-a8e0-41e1c091a660 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:21.571574 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a4e70-00f8-464b-a8e0-41e1c091a660 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:21.651782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:21.679497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a4e70-00f8-464b-a8e0-41e1c091a660 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:21.682126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a4e70-00f8-464b-a8e0-41e1c091a660 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:21.695786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 825/1643] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:23.156910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c1352-4574-4757-a985-a6207c7a0fec None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:23.157430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c1352-4574-4757-a985-a6207c7a0fec None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:23.157525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c1352-4574-4757-a985-a6207c7a0fec None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:23.160071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 819/1644] 10.222.0.57 () {54 vars in 960 bytes} [Thu May 26 21:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:04:23.167097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e532e7-d935-47a9-bb07-e510082277c9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:23.168049 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e532e7-d935-47a9-bb07-e510082277c9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:23.168452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e532e7-d935-47a9-bb07-e510082277c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:23.216074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-32e532e7-d935-47a9-bb07-e510082277c9 None None] MFA Rules not processed for user `4edb53dc871047ce89ebb3b90143a6de`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:23.217945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-32e532e7-d935-47a9-bb07-e510082277c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:23.249855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c1352-4574-4757-a985-a6207c7a0fec None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:23.417092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c1352-4574-4757-a985-a6207c7a0fec None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:23.424027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:23.425475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:23.425763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:23.426038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] PATH_INFO: `/v3/users/324ff7734b0c42869259f8682847b80b` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:23.427336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324ff7734b0c42869259f8682847b80b)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:23.430249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:23.436577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 826/1645] 10.222.0.57 () {58 vars in 1057 bytes} [Thu May 26 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:23.437569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e532e7-d935-47a9-bb07-e510082277c9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:23.447674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:23.476541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e532e7-d935-47a9-bb07-e510082277c9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:23.497653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a3b4-658d-4617-b99e-064546e5531e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:23.503490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a3b4-658d-4617-b99e-064546e5531e service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:23.503792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a3b4-658d-4617-b99e-064546e5531e service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:23.504937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a3b4-658d-4617-b99e-064546e5531e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:23.512222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-caa1a3b4-658d-4617-b99e-064546e5531e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:23.531275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324ff7734b0c42869259f8682847b80b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:23.534639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324ff7734b0c42869259f8682847b80b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:23.558410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324ff7734b0c42869259f8682847b80b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:23.559298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324ff7734b0c42869259f8682847b80b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:23.578129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1a3b4-658d-4617-b99e-064546e5531e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:23.580020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa1a3b4-658d-4617-b99e-064546e5531e service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:23.585570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cde46a52-a739-4d22-86cb-a57eef771d36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324ff7734b0c42869259f8682847b80b'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:23.619971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 820/1646] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:23 2022] DELETE /identity/v3/users/324ff7734b0c42869259f8682847b80b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:23.621109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 827/1647] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:23.622116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bc83f-06e3-4641-aecb-9fd0a20816ac None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:23.622706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bc83f-06e3-4641-aecb-9fd0a20816ac None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:23.623133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bc83f-06e3-4641-aecb-9fd0a20816ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:23.670433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-752bc83f-06e3-4641-aecb-9fd0a20816ac None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:23.671868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-752bc83f-06e3-4641-aecb-9fd0a20816ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:23.847464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 821/1648] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:23.873386 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a3b4-658d-4617-b99e-064546e5531e service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:23.893740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a3b4-658d-4617-b99e-064546e5531e service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:24.072987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e939014-d9da-49e1-844d-9eab02b582f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:24.073678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e939014-d9da-49e1-844d-9eab02b582f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:24.074007 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e939014-d9da-49e1-844d-9eab02b582f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:24.074246 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e939014-d9da-49e1-844d-9eab02b582f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:24.112436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e939014-d9da-49e1-844d-9eab02b582f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:24.115781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e939014-d9da-49e1-844d-9eab02b582f9 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:24.150376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 828/1649] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:25.672389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bc83f-06e3-4641-aecb-9fd0a20816ac None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:25.681145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-752bc83f-06e3-4641-aecb-9fd0a20816ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:25.725139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bc83f-06e3-4641-aecb-9fd0a20816ac None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:25.759513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:25.759685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:25.760097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:25.760453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] PATH_INFO: `/v3/projects/8b4ba0955acb4f48845df8e538376807` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:25.764381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4ba0955acb4f48845df8e538376807)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:25.765606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: WARNING py.warnings [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:04:25.765606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: warnings.warn(msg) May 26 21:04:25.765606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: May 26 21:04:25.767721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:25.934972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4ba0955acb4f48845df8e538376807', 'request_id': 'req-038040e2-3ab5-4806-b1f0-8762ec95c133'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:25.968479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4ba0955acb4f48845df8e538376807', 'request_id': 'req-038040e2-3ab5-4806-b1f0-8762ec95c133'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:26.002110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4ba0955acb4f48845df8e538376807', 'request_id': 'req-038040e2-3ab5-4806-b1f0-8762ec95c133'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:26.041252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] The token cache is being invalidate because project 8b4ba0955acb4f48845df8e538376807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:26.041781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:26.043210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-038040e2-3ab5-4806-b1f0-8762ec95c133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:26.057662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 822/1650] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:25 2022] DELETE /identity/v3/projects/8b4ba0955acb4f48845df8e538376807 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:26.062361 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3e499-2a8c-4e28-babb-48eec3c2de8f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:26.063488 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3e499-2a8c-4e28-babb-48eec3c2de8f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:26.066984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3e499-2a8c-4e28-babb-48eec3c2de8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:26.104930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-83f3e499-2a8c-4e28-babb-48eec3c2de8f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:26.110170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-83f3e499-2a8c-4e28-babb-48eec3c2de8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:26.339795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 829/1651] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:26.342351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c7867-42b4-4bcf-8a37-27f31c1586bd None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:26.342689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c7867-42b4-4bcf-8a37-27f31c1586bd None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:26.343014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c7867-42b4-4bcf-8a37-27f31c1586bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:26.383829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-867c7867-42b4-4bcf-8a37-27f31c1586bd None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:26.390182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-867c7867-42b4-4bcf-8a37-27f31c1586bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:26.425210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 823/1652] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:26.426942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3e499-2a8c-4e28-babb-48eec3c2de8f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:26.434429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:26.441945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3e499-2a8c-4e28-babb-48eec3c2de8f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:26.458745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7102a-5cda-447d-ab84-58bd1efb8e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:26.459970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7102a-5cda-447d-ab84-58bd1efb8e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:26.461517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7102a-5cda-447d-ab84-58bd1efb8e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:26.462071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7102a-5cda-447d-ab84-58bd1efb8e43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:26.463971 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7102a-5cda-447d-ab84-58bd1efb8e43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:26.464729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7102a-5cda-447d-ab84-58bd1efb8e43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:26.471933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7102a-5cda-447d-ab84-58bd1efb8e43 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:26.503335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 830/1653] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:26.504571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c7867-42b4-4bcf-8a37-27f31c1586bd None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:26.511868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:26.540033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c7867-42b4-4bcf-8a37-27f31c1586bd None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:26.570003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f9907-9028-4983-bac2-4d2431131abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:26.572797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f9907-9028-4983-bac2-4d2431131abe admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:26.573206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f9907-9028-4983-bac2-4d2431131abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:26.573598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f9907-9028-4983-bac2-4d2431131abe admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:26.576055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f9907-9028-4983-bac2-4d2431131abe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:26.582138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f9907-9028-4983-bac2-4d2431131abe admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:26.655406 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 824/1654] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:26 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:26.656942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7102a-5cda-447d-ab84-58bd1efb8e43 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:26.680408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7102a-5cda-447d-ab84-58bd1efb8e43 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:26.722899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40612e4-8947-43f3-92e8-db2d5df81713 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:26.724525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40612e4-8947-43f3-92e8-db2d5df81713 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:26.724951 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40612e4-8947-43f3-92e8-db2d5df81713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:26.804563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d40612e4-8947-43f3-92e8-db2d5df81713 None None] MFA Rules not processed for user `83f3cfb67d3147d7b856158dcf32ff03`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:26.805946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d40612e4-8947-43f3-92e8-db2d5df81713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:26.943445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65940e0-0c51-472c-9eb0-c142eb1a8326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:26.944096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65940e0-0c51-472c-9eb0-c142eb1a8326 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:26.944323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65940e0-0c51-472c-9eb0-c142eb1a8326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:26.944609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65940e0-0c51-472c-9eb0-c142eb1a8326 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:26.945329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65940e0-0c51-472c-9eb0-c142eb1a8326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:26.957489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65940e0-0c51-472c-9eb0-c142eb1a8326 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:27.009015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 831/1655] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:26 2022] POST /identity/v3/users => generated 283 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:27.012768 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65940e0-0c51-472c-9eb0-c142eb1a8326 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:27.047828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65940e0-0c51-472c-9eb0-c142eb1a8326 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:27.056545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd63f8-e251-421a-964c-45c8b0f53d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:27.057667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd63f8-e251-421a-964c-45c8b0f53d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:27.057964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd63f8-e251-421a-964c-45c8b0f53d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:27.058297 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd63f8-e251-421a-964c-45c8b0f53d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:27.059685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd63f8-e251-421a-964c-45c8b0f53d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:27.064741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd63f8-e251-421a-964c-45c8b0f53d8e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:27.086695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 832/1656] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:27.089529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd63f8-e251-421a-964c-45c8b0f53d8e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:27.099759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 825/1657] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:27.105629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd63f8-e251-421a-964c-45c8b0f53d8e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:27.120391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7f5ac-b9f2-4a23-85df-9d5d892b90fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:27.121095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7f5ac-b9f2-4a23-85df-9d5d892b90fd admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:27.121386 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7f5ac-b9f2-4a23-85df-9d5d892b90fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:27.121668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7f5ac-b9f2-4a23-85df-9d5d892b90fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:27.122499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7f5ac-b9f2-4a23-85df-9d5d892b90fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:27.126291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7f5ac-b9f2-4a23-85df-9d5d892b90fd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:27.156448 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 833/1658] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:27.157710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40612e4-8947-43f3-92e8-db2d5df81713 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:27.171414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40612e4-8947-43f3-92e8-db2d5df81713 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:27.187221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7d2c8-17fb-43d3-8763-6f24354906a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:27.188241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7d2c8-17fb-43d3-8763-6f24354906a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:27.188649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7d2c8-17fb-43d3-8763-6f24354906a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:27.189073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7d2c8-17fb-43d3-8763-6f24354906a0 admin admin] PATH_INFO: `/v3/projects/60adec237eea4b649a22cdc05315ea4d/users/8af56300164942dfbf83fe275b623488/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:27.245312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7d2c8-17fb-43d3-8763-6f24354906a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60adec237eea4b649a22cdc05315ea4d, user_id=8af56300164942dfbf83fe275b623488, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:27.257083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7d2c8-17fb-43d3-8763-6f24354906a0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:27.280641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7f5ac-b9f2-4a23-85df-9d5d892b90fd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:27.282801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-89f7f5ac-b9f2-4a23-85df-9d5d892b90fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:27.310157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 826/1659] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:27 2022] PUT /identity/v3/projects/60adec237eea4b649a22cdc05315ea4d/users/8af56300164942dfbf83fe275b623488/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:27.325940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcb4e5-8ccb-4ffb-aad4-6ed256f7ba4d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:27.326245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcb4e5-8ccb-4ffb-aad4-6ed256f7ba4d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:27.326556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abcb4e5-8ccb-4ffb-aad4-6ed256f7ba4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:27.451504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3abcb4e5-8ccb-4ffb-aad4-6ed256f7ba4d None None] MFA Rules not processed for user `8af56300164942dfbf83fe275b623488`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:27.452811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3abcb4e5-8ccb-4ffb-aad4-6ed256f7ba4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:27.603144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7f5ac-b9f2-4a23-85df-9d5d892b90fd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:27.727012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 827/1660] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:27.729105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcb4e5-8ccb-4ffb-aad4-6ed256f7ba4d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:27.827890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:27.852448 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:27.883027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a61509-154b-4b4f-aea3-dd52c379d4d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:27.886314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a61509-154b-4b4f-aea3-dd52c379d4d0 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:27.886558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a61509-154b-4b4f-aea3-dd52c379d4d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:27.886895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a61509-154b-4b4f-aea3-dd52c379d4d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:27.906487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abcb4e5-8ccb-4ffb-aad4-6ed256f7ba4d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:27.920270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c1920-0c3a-4cb1-a8aa-11b04800e9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:27.921384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c1920-0c3a-4cb1-a8aa-11b04800e9f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:27.921683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c1920-0c3a-4cb1-a8aa-11b04800e9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:27.922018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c1920-0c3a-4cb1-a8aa-11b04800e9f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:27.922730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c1920-0c3a-4cb1-a8aa-11b04800e9f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:27.942692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c1920-0c3a-4cb1-a8aa-11b04800e9f0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:28.014008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 828/1661] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:27 2022] POST /identity/v3/projects => generated 365 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:28.019868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6ae25-ff2d-4ef7-b99e-4ca04529fc2c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:28.026356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6ae25-ff2d-4ef7-b99e-4ca04529fc2c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:28.026677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6ae25-ff2d-4ef7-b99e-4ca04529fc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:28.027836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a61509-154b-4b4f-aea3-dd52c379d4d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:28.029525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a61509-154b-4b4f-aea3-dd52c379d4d0 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:28.063419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 834/1662] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 783 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:04:28.064474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a61509-154b-4b4f-aea3-dd52c379d4d0 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:28.092051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dba6ae25-ff2d-4ef7-b99e-4ca04529fc2c None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:28.093223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dba6ae25-ff2d-4ef7-b99e-4ca04529fc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:28.107406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a61509-154b-4b4f-aea3-dd52c379d4d0 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:28.227728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 829/1663] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:04:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:28.320466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d160-03aa-4312-af54-39e8a18d60e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:28.321266 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d160-03aa-4312-af54-39e8a18d60e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:28.322503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d160-03aa-4312-af54-39e8a18d60e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:28.325095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274d160-03aa-4312-af54-39e8a18d60e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:28.326014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274d160-03aa-4312-af54-39e8a18d60e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:28.335418 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274d160-03aa-4312-af54-39e8a18d60e2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:28.393532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 835/1664] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:28 2022] POST /identity/v3/users => generated 289 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:28.397559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6ae25-ff2d-4ef7-b99e-4ca04529fc2c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:28.419607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6ae25-ff2d-4ef7-b99e-4ca04529fc2c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:28.458849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f4ccb-9be4-4e23-97b1-7c6dd1dfa2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:28.459556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f4ccb-9be4-4e23-97b1-7c6dd1dfa2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:28.459839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f4ccb-9be4-4e23-97b1-7c6dd1dfa2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:28.460114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f4ccb-9be4-4e23-97b1-7c6dd1dfa2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:28.460853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f4ccb-9be4-4e23-97b1-7c6dd1dfa2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:28.471408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f4ccb-9be4-4e23-97b1-7c6dd1dfa2c7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:28.511137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 830/1665] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:04:28.514209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d160-03aa-4312-af54-39e8a18d60e2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:28.529241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274d160-03aa-4312-af54-39e8a18d60e2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:28.558959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e650409-3431-44a7-b06f-a97d4d1592fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:28.560107 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650409-3431-44a7-b06f-a97d4d1592fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:28.560538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650409-3431-44a7-b06f-a97d4d1592fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:28.560946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e650409-3431-44a7-b06f-a97d4d1592fb admin admin] PATH_INFO: `/v3/projects/38bd7bb43f824cfca0b3922ef3fcc8f5/users/dee28a05b46f47ea8d12c339fadb2d83/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:28.606990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e650409-3431-44a7-b06f-a97d4d1592fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=38bd7bb43f824cfca0b3922ef3fcc8f5, user_id=dee28a05b46f47ea8d12c339fadb2d83, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:28.613379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f4ccb-9be4-4e23-97b1-7c6dd1dfa2c7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:28.616819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-822f4ccb-9be4-4e23-97b1-7c6dd1dfa2c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:28.623743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e650409-3431-44a7-b06f-a97d4d1592fb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:28.689108 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 836/1666] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:28 2022] PUT /identity/v3/projects/38bd7bb43f824cfca0b3922ef3fcc8f5/users/dee28a05b46f47ea8d12c339fadb2d83/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:28.695526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e650409-3431-44a7-b06f-a97d4d1592fb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:28.835115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f4ccb-9be4-4e23-97b1-7c6dd1dfa2c7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:28.864558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e650409-3431-44a7-b06f-a97d4d1592fb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:28.878351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d47651-6b5f-4fb9-9637-134bc80d2cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:28.879168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d47651-6b5f-4fb9-9637-134bc80d2cdd admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:28.879459 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d47651-6b5f-4fb9-9637-134bc80d2cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:28.879745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d47651-6b5f-4fb9-9637-134bc80d2cdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:28.880494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d47651-6b5f-4fb9-9637-134bc80d2cdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:28.882205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d47651-6b5f-4fb9-9637-134bc80d2cdd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:28.884146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:28.916701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 837/1667] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:04:28.917837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d47651-6b5f-4fb9-9637-134bc80d2cdd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:28.955113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d47651-6b5f-4fb9-9637-134bc80d2cdd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:28.967449 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea11ef-403c-49d7-af7f-97d5e47f9dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:28.969993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea11ef-403c-49d7-af7f-97d5e47f9dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:28.970420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea11ef-403c-49d7-af7f-97d5e47f9dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:28.970639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea11ef-403c-49d7-af7f-97d5e47f9dcf admin admin] PATH_INFO: `/v3/projects/38bd7bb43f824cfca0b3922ef3fcc8f5/users/dee28a05b46f47ea8d12c339fadb2d83/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:28.978062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea11ef-403c-49d7-af7f-97d5e47f9dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=38bd7bb43f824cfca0b3922ef3fcc8f5, user_id=dee28a05b46f47ea8d12c339fadb2d83, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:28.980905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea11ef-403c-49d7-af7f-97d5e47f9dcf admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:28.984653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:28.996004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29af020-5154-41ec-8d1f-ffbd80941ea5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:28.996745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29af020-5154-41ec-8d1f-ffbd80941ea5 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:28.997083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29af020-5154-41ec-8d1f-ffbd80941ea5 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:28.997383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29af020-5154-41ec-8d1f-ffbd80941ea5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:29.009728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 838/1668] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:28 2022] PUT /identity/v3/projects/38bd7bb43f824cfca0b3922ef3fcc8f5/users/dee28a05b46f47ea8d12c339fadb2d83/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:29.021405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248195e-ccb6-4d55-945e-7cc94096821d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:29.021737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248195e-ccb6-4d55-945e-7cc94096821d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:29.021977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248195e-ccb6-4d55-945e-7cc94096821d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:29.090072 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c248195e-ccb6-4d55-945e-7cc94096821d None None] MFA Rules not processed for user `dee28a05b46f47ea8d12c339fadb2d83`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:29.091494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c248195e-ccb6-4d55-945e-7cc94096821d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:29.133883 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29af020-5154-41ec-8d1f-ffbd80941ea5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:29.135632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29af020-5154-41ec-8d1f-ffbd80941ea5 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:29.159057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 831/1669] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:04:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 21:04:29.235321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 839/1670] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:29.246251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29af020-5154-41ec-8d1f-ffbd80941ea5 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:29.248035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b29af020-5154-41ec-8d1f-ffbd80941ea5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:29.374604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29af020-5154-41ec-8d1f-ffbd80941ea5 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:29.436387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf7438-71d4-44bf-80ec-4a9e45663359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:29.437192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf7438-71d4-44bf-80ec-4a9e45663359 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:29.437485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf7438-71d4-44bf-80ec-4a9e45663359 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:29.437765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf7438-71d4-44bf-80ec-4a9e45663359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:29.450161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf7438-71d4-44bf-80ec-4a9e45663359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:29.452063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf7438-71d4-44bf-80ec-4a9e45663359 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:29.464382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 832/1671] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:29.974336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248195e-ccb6-4d55-945e-7cc94096821d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:29.977762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c248195e-ccb6-4d55-945e-7cc94096821d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:30.306019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248195e-ccb6-4d55-945e-7cc94096821d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:30.319667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:30.320734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:30.321127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:30.321518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] PATH_INFO: `/v3/users/a0c0eb6b15754e71908441206e7be36c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:30.324296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0c0eb6b15754e71908441206e7be36c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:30.325658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: WARNING py.warnings [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:04:30.325658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: warnings.warn(msg) May 26 21:04:30.325658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: May 26 21:04:30.328772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:30.538494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0c0eb6b15754e71908441206e7be36c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:30.589344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0c0eb6b15754e71908441206e7be36c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:30.595999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:30.603511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0c0eb6b15754e71908441206e7be36c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:30.604546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0c0eb6b15754e71908441206e7be36c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:30.644635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0c0eb6b15754e71908441206e7be36c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:30.665544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 840/1672] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:29 2022] DELETE /identity/v3/users/a0c0eb6b15754e71908441206e7be36c => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:30.667314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbfd6a-ddfe-4cba-81c8-aae1b1ce951a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:30.667805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbfd6a-ddfe-4cba-81c8-aae1b1ce951a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:30.667950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbfd6a-ddfe-4cba-81c8-aae1b1ce951a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:30.760972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-25fbfd6a-ddfe-4cba-81c8-aae1b1ce951a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:30.762229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-25fbfd6a-ddfe-4cba-81c8-aae1b1ce951a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:30.801601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:30.804259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:31.010919 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1c2af-804b-4b53-bf0e-16d67fb55015 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:31.018897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:31.033820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:31.034866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:31.035151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:31.036354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] PATH_INFO: `/v3/users/8af56300164942dfbf83fe275b623488` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:31.041802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 833/1673] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:04:31.043649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af56300164942dfbf83fe275b623488)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:31.058628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbfd6a-ddfe-4cba-81c8-aae1b1ce951a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:31.062823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:31.064220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-25fbfd6a-ddfe-4cba-81c8-aae1b1ce951a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:31.270084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af56300164942dfbf83fe275b623488'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:31.305345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af56300164942dfbf83fe275b623488'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:31.315679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fbfd6a-ddfe-4cba-81c8-aae1b1ce951a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:31.322409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af56300164942dfbf83fe275b623488'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:31.323897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af56300164942dfbf83fe275b623488'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:31.326199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3da1d-67f9-4d4c-8cc9-1b8ce99b54c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:31.330820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3da1d-67f9-4d4c-8cc9-1b8ce99b54c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:31.331534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3da1d-67f9-4d4c-8cc9-1b8ce99b54c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:31.333178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3da1d-67f9-4d4c-8cc9-1b8ce99b54c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:31.337020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3ed3da1d-67f9-4d4c-8cc9-1b8ce99b54c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:31.340934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9b4b20fc-8c41-4727-9561-2ea600eb83fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af56300164942dfbf83fe275b623488'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:31.359452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 841/1674] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:30 2022] DELETE /identity/v3/users/8af56300164942dfbf83fe275b623488 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:31.361716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be161d-24ff-4830-891f-ec5974770ea3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:31.362147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be161d-24ff-4830-891f-ec5974770ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:31.362607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be161d-24ff-4830-891f-ec5974770ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:31.388884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3da1d-67f9-4d4c-8cc9-1b8ce99b54c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:31.394046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3da1d-67f9-4d4c-8cc9-1b8ce99b54c9 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:31.400714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-50be161d-24ff-4830-891f-ec5974770ea3 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:31.424448 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-50be161d-24ff-4830-891f-ec5974770ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:31.460399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 834/1675] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:31.712922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 842/1676] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:31.821966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3da1d-67f9-4d4c-8cc9-1b8ce99b54c9 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:31.823048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3ed3da1d-67f9-4d4c-8cc9-1b8ce99b54c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:31.846718 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3da1d-67f9-4d4c-8cc9-1b8ce99b54c9 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:32.128406 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156953cf-1d4c-4200-a050-69183d44a3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:32.129257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156953cf-1d4c-4200-a050-69183d44a3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:32.132470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156953cf-1d4c-4200-a050-69183d44a3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:32.133643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156953cf-1d4c-4200-a050-69183d44a3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:32.170497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156953cf-1d4c-4200-a050-69183d44a3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:32.181059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156953cf-1d4c-4200-a050-69183d44a3b9 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:32.223246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 835/1677] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:33.376696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be161d-24ff-4830-891f-ec5974770ea3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:33.383236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-50be161d-24ff-4830-891f-ec5974770ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:33.423569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be161d-24ff-4830-891f-ec5974770ea3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:33.449688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:33.450635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:33.450925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:33.451227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] PATH_INFO: `/v3/projects/2aeed4619d4f4528ba73300df3d7a07d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:33.461424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aeed4619d4f4528ba73300df3d7a07d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:33.463607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:33.465618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:33.690316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aeed4619d4f4528ba73300df3d7a07d', 'request_id': 'req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:33.702640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156953cf-1d4c-4200-a050-69183d44a3b9 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:33.706868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-156953cf-1d4c-4200-a050-69183d44a3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:33.708953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aeed4619d4f4528ba73300df3d7a07d', 'request_id': 'req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:33.735160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aeed4619d4f4528ba73300df3d7a07d', 'request_id': 'req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:33.736577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156953cf-1d4c-4200-a050-69183d44a3b9 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:33.766258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] The token cache is being invalidate because project 2aeed4619d4f4528ba73300df3d7a07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:33.766593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:33.772374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ffb0b0fa-b86b-48e5-a0a9-76e0e4df0dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:33.777476 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 843/1678] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:33 2022] DELETE /identity/v3/projects/2aeed4619d4f4528ba73300df3d7a07d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:33.779184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f7599-fe2b-48b2-af85-4191c27a734d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:33.779476 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f7599-fe2b-48b2-af85-4191c27a734d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:33.779767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f7599-fe2b-48b2-af85-4191c27a734d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:33.859021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-563f7599-fe2b-48b2-af85-4191c27a734d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:33.860990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-563f7599-fe2b-48b2-af85-4191c27a734d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:33.973638 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:33.974562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:33.974942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:33.975230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] PATH_INFO: `/v3/users/1168b6a97b30483f8e3aed805200d7c5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:33.980781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1168b6a97b30483f8e3aed805200d7c5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:33.983059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:34.098542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1168b6a97b30483f8e3aed805200d7c5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:34.099847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1168b6a97b30483f8e3aed805200d7c5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:34.102253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:34.120905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1168b6a97b30483f8e3aed805200d7c5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:34.123676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 844/1679] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:04:34.127920 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1168b6a97b30483f8e3aed805200d7c5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:34.131465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cad97-5d2b-4438-906b-a1d8d62c5215 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:34.131465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cad97-5d2b-4438-906b-a1d8d62c5215 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:34.131704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cad97-5d2b-4438-906b-a1d8d62c5215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:34.181607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c0e68495-0c02-4d04-aebb-00c76edfb3f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1168b6a97b30483f8e3aed805200d7c5'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:34.205529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b60cad97-5d2b-4438-906b-a1d8d62c5215 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:34.209171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b60cad97-5d2b-4438-906b-a1d8d62c5215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:34.230550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 836/1680] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:33 2022] DELETE /identity/v3/users/1168b6a97b30483f8e3aed805200d7c5 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:34.238609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356a6a7-80d5-4187-9761-aa7730717189 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:34.238959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356a6a7-80d5-4187-9761-aa7730717189 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:34.239248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356a6a7-80d5-4187-9761-aa7730717189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:34.252588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 845/1681] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:34.257571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cad97-5d2b-4438-906b-a1d8d62c5215 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:34.279457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cad97-5d2b-4438-906b-a1d8d62c5215 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:34.282763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1356a6a7-80d5-4187-9761-aa7730717189 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:34.287272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1356a6a7-80d5-4187-9761-aa7730717189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:34.487813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47917380-603c-490a-801d-3ed37967706e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:34.490239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47917380-603c-490a-801d-3ed37967706e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:34.491627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47917380-603c-490a-801d-3ed37967706e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:34.492051 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47917380-603c-490a-801d-3ed37967706e admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:34.497041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47917380-603c-490a-801d-3ed37967706e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:34.502483 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47917380-603c-490a-801d-3ed37967706e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:34.505499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47917380-603c-490a-801d-3ed37967706e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:34.506668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 837/1682] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:04:34.551360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 846/1683] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:34.552084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356a6a7-80d5-4187-9761-aa7730717189 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:34.580223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356a6a7-80d5-4187-9761-aa7730717189 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:34.590455 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b325f6-c9be-454b-b721-edb5df2bf042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:34.591206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b325f6-c9be-454b-b721-edb5df2bf042 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:34.591636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b325f6-c9be-454b-b721-edb5df2bf042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:34.591742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b325f6-c9be-454b-b721-edb5df2bf042 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:34.592441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b325f6-c9be-454b-b721-edb5df2bf042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:34.592934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47917380-603c-490a-801d-3ed37967706e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:34.594387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b325f6-c9be-454b-b721-edb5df2bf042 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:34.594959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-47917380-603c-490a-801d-3ed37967706e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:34.634116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:34.686947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 838/1684] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:34 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:04:34.691649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d80d3-7ca5-491c-b480-1934a92ee4c4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:34.692232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d80d3-7ca5-491c-b480-1934a92ee4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:34.692734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d80d3-7ca5-491c-b480-1934a92ee4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:34.739102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f15d80d3-7ca5-491c-b480-1934a92ee4c4 None None] MFA Rules not processed for user `94b8d8b089264b5192a651b41bf2a205`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:34.741083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f15d80d3-7ca5-491c-b480-1934a92ee4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:34.748652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47917380-603c-490a-801d-3ed37967706e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:34.897267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 839/1685] 10.222.0.62 () {58 vars in 1015 bytes} [Thu May 26 21:04:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:34.898130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d80d3-7ca5-491c-b480-1934a92ee4c4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:34.917944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb5174-5e41-4104-a735-3959c31afe63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:34.920816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb5174-5e41-4104-a735-3959c31afe63 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:34.921863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb5174-5e41-4104-a735-3959c31afe63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:34.922798 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb5174-5e41-4104-a735-3959c31afe63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:35.018115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:35.042962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d80d3-7ca5-491c-b480-1934a92ee4c4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:35.054846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3198b2d-3c40-4804-85ea-85ccf7da5706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:35.055836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3198b2d-3c40-4804-85ea-85ccf7da5706 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:35.056222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3198b2d-3c40-4804-85ea-85ccf7da5706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:35.056695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3198b2d-3c40-4804-85ea-85ccf7da5706 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:35.057586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3198b2d-3c40-4804-85ea-85ccf7da5706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:35.059711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3198b2d-3c40-4804-85ea-85ccf7da5706 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:35.114576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 840/1686] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:34 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:35.114576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3198b2d-3c40-4804-85ea-85ccf7da5706 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:35.116984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f3198b2d-3c40-4804-85ea-85ccf7da5706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:35.145800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3198b2d-3c40-4804-85ea-85ccf7da5706 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:35.150225 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708459-ab54-4a72-9c20-3541ef7488d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:35.151053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708459-ab54-4a72-9c20-3541ef7488d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:35.151440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708459-ab54-4a72-9c20-3541ef7488d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:35.151820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9708459-ab54-4a72-9c20-3541ef7488d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:35.171017 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9708459-ab54-4a72-9c20-3541ef7488d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:35.172655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9708459-ab54-4a72-9c20-3541ef7488d8 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:35.174295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb5174-5e41-4104-a735-3959c31afe63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:35.180420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb5174-5e41-4104-a735-3959c31afe63 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:35.206640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 841/1687] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:35.207775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708459-ab54-4a72-9c20-3541ef7488d8 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:35.209373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b9708459-ab54-4a72-9c20-3541ef7488d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:35.221220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 847/1688] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:04:35.225162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18443f57-ffd0-4754-a902-a529415fac9b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:35.227166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18443f57-ffd0-4754-a902-a529415fac9b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:35.227769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18443f57-ffd0-4754-a902-a529415fac9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:35.238108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9708459-ab54-4a72-9c20-3541ef7488d8 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:35.250933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:35.253138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:35.253531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:35.254483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] PATH_INFO: `/v3/projects/60adec237eea4b649a22cdc05315ea4d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:35.259308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=60adec237eea4b649a22cdc05315ea4d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:35.261589 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:35.326862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-18443f57-ffd0-4754-a902-a529415fac9b None None] MFA Rules not processed for user `8145168b1d02438886cd0865844f7480`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:35.331822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-18443f57-ffd0-4754-a902-a529415fac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:35.392089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60adec237eea4b649a22cdc05315ea4d', 'request_id': 'req-847d96ad-eff2-4548-93a9-1385e03bd1dc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:35.403253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60adec237eea4b649a22cdc05315ea4d', 'request_id': 'req-847d96ad-eff2-4548-93a9-1385e03bd1dc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:35.418571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60adec237eea4b649a22cdc05315ea4d', 'request_id': 'req-847d96ad-eff2-4548-93a9-1385e03bd1dc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:35.444246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] The token cache is being invalidate because project 60adec237eea4b649a22cdc05315ea4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:35.444811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:35.445665 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:35.450325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 842/1689] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:35 2022] DELETE /identity/v3/projects/60adec237eea4b649a22cdc05315ea4d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:35.451388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:35.455056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:35.600899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d96ad-eff2-4548-93a9-1385e03bd1dc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:35.613008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 848/1690] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:35.616836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18443f57-ffd0-4754-a902-a529415fac9b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:35.619727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c1762-4bc6-4d22-bb78-1da54547bf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:35.620604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c1762-4bc6-4d22-bb78-1da54547bf78 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:35.620898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c1762-4bc6-4d22-bb78-1da54547bf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:35.621675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86c1762-4bc6-4d22-bb78-1da54547bf78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:35.621947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c1762-4bc6-4d22-bb78-1da54547bf78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:35.623548 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-18443f57-ffd0-4754-a902-a529415fac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:35.626562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86c1762-4bc6-4d22-bb78-1da54547bf78 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:35.672236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 843/1691] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:35.673145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c1762-4bc6-4d22-bb78-1da54547bf78 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:35.681934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b86c1762-4bc6-4d22-bb78-1da54547bf78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:35.696258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18443f57-ffd0-4754-a902-a529415fac9b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:35.710397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:35.713875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:35.713875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:35.713875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] PATH_INFO: `/v3/users/dee28a05b46f47ea8d12c339fadb2d83` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:35.718809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dee28a05b46f47ea8d12c339fadb2d83)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:35.724860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:35.746994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:35.832888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:35.912518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dee28a05b46f47ea8d12c339fadb2d83'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:35.919409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86c1762-4bc6-4d22-bb78-1da54547bf78 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:35.938175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dee28a05b46f47ea8d12c339fadb2d83'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:35.951223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfb52a-149a-44a3-8371-2eb975127848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:35.952684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfb52a-149a-44a3-8371-2eb975127848 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:35.953149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfb52a-149a-44a3-8371-2eb975127848 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:35.953438 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfb52a-149a-44a3-8371-2eb975127848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:35.958939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dee28a05b46f47ea8d12c339fadb2d83'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:35.959602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dee28a05b46f47ea8d12c339fadb2d83'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:35.981217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dee28a05b46f47ea8d12c339fadb2d83'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:36.004709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 849/1692] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:35 2022] DELETE /identity/v3/users/dee28a05b46f47ea8d12c339fadb2d83 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:36.008364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:36.020426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:36.098791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfb52a-149a-44a3-8371-2eb975127848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:36.098791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfb52a-149a-44a3-8371-2eb975127848 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:36.125275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 844/1693] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:36.246699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11df3eb-9376-488c-92d3-67ffc0c4b07e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:36.261422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f53fa4-e13e-4f45-b631-c09b9794e94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:36.262132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f53fa4-e13e-4f45-b631-c09b9794e94a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:36.262417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f53fa4-e13e-4f45-b631-c09b9794e94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:36.262756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f53fa4-e13e-4f45-b631-c09b9794e94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:36.263533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f53fa4-e13e-4f45-b631-c09b9794e94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:36.266545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f53fa4-e13e-4f45-b631-c09b9794e94a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:36.302941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 850/1694] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:36 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:36.307233 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfb52a-149a-44a3-8371-2eb975127848 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:36.333519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfb52a-149a-44a3-8371-2eb975127848 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:36.345933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2622170-a6b8-4328-a682-d12f6065bc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:36.346857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2622170-a6b8-4328-a682-d12f6065bc1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:36.347148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2622170-a6b8-4328-a682-d12f6065bc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:36.347431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2622170-a6b8-4328-a682-d12f6065bc1d admin admin] PATH_INFO: `/v3/projects/19d4ead6e092498daaf1d4ddb55a7653/users/f2222e0928d3481e97646167edf90f8b/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:36.387357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2622170-a6b8-4328-a682-d12f6065bc1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d4ead6e092498daaf1d4ddb55a7653, user_id=f2222e0928d3481e97646167edf90f8b, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:36.390095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2622170-a6b8-4328-a682-d12f6065bc1d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:36.438159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 845/1695] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:36 2022] PUT /identity/v3/projects/19d4ead6e092498daaf1d4ddb55a7653/users/f2222e0928d3481e97646167edf90f8b/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:36.441121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef2e6a-3df7-4cf4-9bae-68f4fa207f97 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:36.441121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef2e6a-3df7-4cf4-9bae-68f4fa207f97 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:36.441446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef2e6a-3df7-4cf4-9bae-68f4fa207f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:36.511455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-91ef2e6a-3df7-4cf4-9bae-68f4fa207f97 None None] MFA Rules not processed for user `f2222e0928d3481e97646167edf90f8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:36.512704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-91ef2e6a-3df7-4cf4-9bae-68f4fa207f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:36.669828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 851/1696] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:36.671532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2622170-a6b8-4328-a682-d12f6065bc1d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:36.846664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2622170-a6b8-4328-a682-d12f6065bc1d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:36.865998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e7d6c-2d4c-4528-9ff7-3027c8bac6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:36.866803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7d6c-2d4c-4528-9ff7-3027c8bac6ca admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:36.867093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7d6c-2d4c-4528-9ff7-3027c8bac6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:36.870402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7d6c-2d4c-4528-9ff7-3027c8bac6ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:36.870402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e7d6c-2d4c-4528-9ff7-3027c8bac6ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:36.870402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e7d6c-2d4c-4528-9ff7-3027c8bac6ca admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:36.928384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 846/1697] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:36 2022] POST /identity/v3/projects => generated 371 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:04:36.929873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef2e6a-3df7-4cf4-9bae-68f4fa207f97 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:36.942364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef2e6a-3df7-4cf4-9bae-68f4fa207f97 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:36.980999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:37.198838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284422e-a54a-4a6f-8edd-a26ec8a9798c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:37.201146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284422e-a54a-4a6f-8edd-a26ec8a9798c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:37.201469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284422e-a54a-4a6f-8edd-a26ec8a9798c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:37.206272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8284422e-a54a-4a6f-8edd-a26ec8a9798c admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:37.207567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284422e-a54a-4a6f-8edd-a26ec8a9798c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:37.209604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8284422e-a54a-4a6f-8edd-a26ec8a9798c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:37.266013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 852/1698] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:36 2022] POST /identity/v3/users => generated 292 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:37.270187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e7d6c-2d4c-4528-9ff7-3027c8bac6ca admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:37.282592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e7d6c-2d4c-4528-9ff7-3027c8bac6ca admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:37.330811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3430-ba5f-4a84-9fcc-46ff99abc143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:37.331561 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3430-ba5f-4a84-9fcc-46ff99abc143 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:37.331849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3430-ba5f-4a84-9fcc-46ff99abc143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:37.332131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e3430-ba5f-4a84-9fcc-46ff99abc143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:37.332896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e3430-ba5f-4a84-9fcc-46ff99abc143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:37.334607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e3430-ba5f-4a84-9fcc-46ff99abc143 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:37.389784 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 847/1699] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:04:37.393666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284422e-a54a-4a6f-8edd-a26ec8a9798c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:37.413252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8284422e-a54a-4a6f-8edd-a26ec8a9798c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:37.430125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a881fe7-4643-4dbc-9fa3-bccecff01b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:37.431193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a881fe7-4643-4dbc-9fa3-bccecff01b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:37.431601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a881fe7-4643-4dbc-9fa3-bccecff01b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:37.431899 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a881fe7-4643-4dbc-9fa3-bccecff01b28 admin admin] PATH_INFO: `/v3/projects/5fa25ced23894e05b224a0acd5bc144c/users/35c3d92af9c84f30a072877e0f37b356/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:37.460147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a881fe7-4643-4dbc-9fa3-bccecff01b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa25ced23894e05b224a0acd5bc144c, user_id=35c3d92af9c84f30a072877e0f37b356, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:37.467334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a881fe7-4643-4dbc-9fa3-bccecff01b28 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:37.493048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 853/1700] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:37 2022] PUT /identity/v3/projects/5fa25ced23894e05b224a0acd5bc144c/users/35c3d92af9c84f30a072877e0f37b356/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:37.495524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3430-ba5f-4a84-9fcc-46ff99abc143 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:37.512827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e3430-ba5f-4a84-9fcc-46ff99abc143 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:37.596464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a881fe7-4643-4dbc-9fa3-bccecff01b28 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:37.598485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0a881fe7-4643-4dbc-9fa3-bccecff01b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:37.665046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:37.751046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afb7a9-08e0-4f8a-9e33-909b409a5513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:37.753207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afb7a9-08e0-4f8a-9e33-909b409a5513 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:37.753664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afb7a9-08e0-4f8a-9e33-909b409a5513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:37.754971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72afb7a9-08e0-4f8a-9e33-909b409a5513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:37.756059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afb7a9-08e0-4f8a-9e33-909b409a5513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:37.761042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72afb7a9-08e0-4f8a-9e33-909b409a5513 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:37.808270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 848/1701] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:37 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:37.813452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afb7a9-08e0-4f8a-9e33-909b409a5513 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:37.820737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-72afb7a9-08e0-4f8a-9e33-909b409a5513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:37.859555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72afb7a9-08e0-4f8a-9e33-909b409a5513 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:37.865896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:37.866983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:37.867429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:37.867833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] PATH_INFO: `/v3/projects/38bd7bb43f824cfca0b3922ef3fcc8f5` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:37.869310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38bd7bb43f824cfca0b3922ef3fcc8f5)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:37.872676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:37.899859 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a881fe7-4643-4dbc-9fa3-bccecff01b28 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:37.935458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:37.936437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:37.936985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:37.943949 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] PATH_INFO: `/v3/projects/e3fdd66858554009b0ccd42d6b8d11f5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:37.947653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3fdd66858554009b0ccd42d6b8d11f5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:37.963675 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:37.995600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38bd7bb43f824cfca0b3922ef3fcc8f5', 'request_id': 'req-3dbb04de-4f60-4f22-9197-c9643a0a0b97'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:38.004612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38bd7bb43f824cfca0b3922ef3fcc8f5', 'request_id': 'req-3dbb04de-4f60-4f22-9197-c9643a0a0b97'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:38.032035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38bd7bb43f824cfca0b3922ef3fcc8f5', 'request_id': 'req-3dbb04de-4f60-4f22-9197-c9643a0a0b97'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:38.065732 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] The token cache is being invalidate because project 38bd7bb43f824cfca0b3922ef3fcc8f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:38.066228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:38.066847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:38.068672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 849/1702] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:37 2022] DELETE /identity/v3/projects/38bd7bb43f824cfca0b3922ef3fcc8f5 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:38.073581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:38.089573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:38.159859 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3fdd66858554009b0ccd42d6b8d11f5', 'request_id': 'req-ff4cf891-3f59-4dbc-b240-b8c396bda636'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:38.202601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3fdd66858554009b0ccd42d6b8d11f5', 'request_id': 'req-ff4cf891-3f59-4dbc-b240-b8c396bda636'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:38.242083 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3fdd66858554009b0ccd42d6b8d11f5', 'request_id': 'req-ff4cf891-3f59-4dbc-b240-b8c396bda636'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:38.301151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] The token cache is being invalidate because project e3fdd66858554009b0ccd42d6b8d11f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:38.304763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:38.310942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ff4cf891-3f59-4dbc-b240-b8c396bda636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:38.322552 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 854/1703] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:37 2022] DELETE /identity/v3/projects/e3fdd66858554009b0ccd42d6b8d11f5 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:38.333634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edb4e2-2262-4cdf-9c5d-3bf56bd28bca None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:38.334235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edb4e2-2262-4cdf-9c5d-3bf56bd28bca None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:38.334689 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edb4e2-2262-4cdf-9c5d-3bf56bd28bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:38.374925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:38.380265 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb04de-4f60-4f22-9197-c9643a0a0b97 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:38.398773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d6edb4e2-2262-4cdf-9c5d-3bf56bd28bca None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:38.407272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d6edb4e2-2262-4cdf-9c5d-3bf56bd28bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:38.539127 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ce4f2-ab21-4a68-ae31-0ef2fd8d6df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:38.541151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ce4f2-ab21-4a68-ae31-0ef2fd8d6df1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:38.542737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ce4f2-ab21-4a68-ae31-0ef2fd8d6df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:38.543266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ce4f2-ab21-4a68-ae31-0ef2fd8d6df1 admin admin] PATH_INFO: `/v3/projects/5fa25ced23894e05b224a0acd5bc144c/users/35c3d92af9c84f30a072877e0f37b356/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:38.551234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ce4f2-ab21-4a68-ae31-0ef2fd8d6df1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa25ced23894e05b224a0acd5bc144c, user_id=35c3d92af9c84f30a072877e0f37b356, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:38.557466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ce4f2-ab21-4a68-ae31-0ef2fd8d6df1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:38.576607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:38.598971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 850/1704] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:38 2022] PUT /identity/v3/projects/5fa25ced23894e05b224a0acd5bc144c/users/35c3d92af9c84f30a072877e0f37b356/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:38.610741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124e216-6c2c-4d54-a1b7-6432809d91a9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:38.611653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124e216-6c2c-4d54-a1b7-6432809d91a9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:38.611903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124e216-6c2c-4d54-a1b7-6432809d91a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:38.640272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 855/1705] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:38.649205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeba98d-4bf9-4d76-8cea-0ba42e42a0fa None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:38.649508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeba98d-4bf9-4d76-8cea-0ba42e42a0fa None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:38.649777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeba98d-4bf9-4d76-8cea-0ba42e42a0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:38.651314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b124e216-6c2c-4d54-a1b7-6432809d91a9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:38.652735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b124e216-6c2c-4d54-a1b7-6432809d91a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:38.739991 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bfeba98d-4bf9-4d76-8cea-0ba42e42a0fa None None] MFA Rules not processed for user `35c3d92af9c84f30a072877e0f37b356`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:38.741113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bfeba98d-4bf9-4d76-8cea-0ba42e42a0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:38.914511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 851/1706] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:38.922557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994062f0-0c3b-4b22-9370-dc65b7f89601 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:38.923023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994062f0-0c3b-4b22-9370-dc65b7f89601 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:38.924010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994062f0-0c3b-4b22-9370-dc65b7f89601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:38.954208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 856/1707] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:38.956924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a74f7e-94b0-41f5-86cc-1221b6001917 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:38.957501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a74f7e-94b0-41f5-86cc-1221b6001917 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:38.957926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a74f7e-94b0-41f5-86cc-1221b6001917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:38.987547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-994062f0-0c3b-4b22-9370-dc65b7f89601 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:39.000341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-994062f0-0c3b-4b22-9370-dc65b7f89601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:39.035494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-85a74f7e-94b0-41f5-86cc-1221b6001917 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:39.037692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-85a74f7e-94b0-41f5-86cc-1221b6001917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:39.044715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 852/1708] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:39.045758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994062f0-0c3b-4b22-9370-dc65b7f89601 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:39.049408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-994062f0-0c3b-4b22-9370-dc65b7f89601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:39.051465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 857/1709] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:39.052631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a74f7e-94b0-41f5-86cc-1221b6001917 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:39.073091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a74f7e-94b0-41f5-86cc-1221b6001917 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:39.084988 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8138b91-60d7-4c52-b452-cef3bb31a0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:39.090020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8138b91-60d7-4c52-b452-cef3bb31a0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:39.090549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8138b91-60d7-4c52-b452-cef3bb31a0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:39.091577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8138b91-60d7-4c52-b452-cef3bb31a0e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:39.092652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8138b91-60d7-4c52-b452-cef3bb31a0e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:39.094114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8138b91-60d7-4c52-b452-cef3bb31a0e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:39.101823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8138b91-60d7-4c52-b452-cef3bb31a0e7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:39.147855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 858/1710] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:39.147855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8138b91-60d7-4c52-b452-cef3bb31a0e7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:39.184506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8138b91-60d7-4c52-b452-cef3bb31a0e7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:39.206041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd1fed-8cf0-4c65-bfb6-d222f55374c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:39.206770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd1fed-8cf0-4c65-bfb6-d222f55374c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:39.207068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd1fed-8cf0-4c65-bfb6-d222f55374c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:39.207888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd1fed-8cf0-4c65-bfb6-d222f55374c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:39.208080 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd1fed-8cf0-4c65-bfb6-d222f55374c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:39.212010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd1fed-8cf0-4c65-bfb6-d222f55374c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:39.218167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd1fed-8cf0-4c65-bfb6-d222f55374c0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:39.270013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 859/1711] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:39.272941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd1fed-8cf0-4c65-bfb6-d222f55374c0 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:39.273330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994062f0-0c3b-4b22-9370-dc65b7f89601 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:39.296327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c2928-af76-4f7e-ba17-f4e90a4c273b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:39.299056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c2928-af76-4f7e-ba17-f4e90a4c273b service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:39.300422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c2928-af76-4f7e-ba17-f4e90a4c273b service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:39.300860 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c2928-af76-4f7e-ba17-f4e90a4c273b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:39.311552 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd1fed-8cf0-4c65-bfb6-d222f55374c0 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:39.321690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c2928-af76-4f7e-ba17-f4e90a4c273b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:39.324973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc969fd-3158-4fc8-9e68-66a1a3e8dd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:39.325976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c2928-af76-4f7e-ba17-f4e90a4c273b service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:39.329281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc969fd-3158-4fc8-9e68-66a1a3e8dd38 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:39.330760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc969fd-3158-4fc8-9e68-66a1a3e8dd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:39.331312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc969fd-3158-4fc8-9e68-66a1a3e8dd38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:39.334228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc969fd-3158-4fc8-9e68-66a1a3e8dd38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:39.338018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc969fd-3158-4fc8-9e68-66a1a3e8dd38 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:39.356858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 853/1712] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:39.357950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c2928-af76-4f7e-ba17-f4e90a4c273b service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:39.408868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c2928-af76-4f7e-ba17-f4e90a4c273b service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:39.420843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 860/1713] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:39 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:39.422331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c827afc-9264-474e-b3c8-a8d10995894b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:39.423364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c827afc-9264-474e-b3c8-a8d10995894b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:39.432227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc969fd-3158-4fc8-9e68-66a1a3e8dd38 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:39.435105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c827afc-9264-474e-b3c8-a8d10995894b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:39.435399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c827afc-9264-474e-b3c8-a8d10995894b admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:39.438398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c827afc-9264-474e-b3c8-a8d10995894b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:39.445001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c827afc-9264-474e-b3c8-a8d10995894b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:39.538007 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 854/1714] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:39 2022] POST /identity/v3/projects => generated 403 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:39.539805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c827afc-9264-474e-b3c8-a8d10995894b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:39.592786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:39.702003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc969fd-3158-4fc8-9e68-66a1a3e8dd38 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:39.761296 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c827afc-9264-474e-b3c8-a8d10995894b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:39.779582 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9afddb-df9d-4c7a-8e09-4069561b5f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:39.780348 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9afddb-df9d-4c7a-8e09-4069561b5f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:39.780638 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9afddb-df9d-4c7a-8e09-4069561b5f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:39.780919 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9afddb-df9d-4c7a-8e09-4069561b5f2b admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:39.781626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9afddb-df9d-4c7a-8e09-4069561b5f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:39.804959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9afddb-df9d-4c7a-8e09-4069561b5f2b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:39.841341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:39.862997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 855/1715] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:39 2022] POST /identity/v3/users => generated 302 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:39.864091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9afddb-df9d-4c7a-8e09-4069561b5f2b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:39.949525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbec356-5bb9-43f2-814d-49480616a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:39.950222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbec356-5bb9-43f2-814d-49480616a553 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:39.950754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbec356-5bb9-43f2-814d-49480616a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:39.951190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbec356-5bb9-43f2-814d-49480616a553 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:39.952429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbec356-5bb9-43f2-814d-49480616a553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:39.954411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbec356-5bb9-43f2-814d-49480616a553 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:40.035429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 861/1716] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:39 2022] POST /identity/v3/users => generated 289 bytes in 611 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:40.044844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbec356-5bb9-43f2-814d-49480616a553 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:40.087574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9afddb-df9d-4c7a-8e09-4069561b5f2b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:40.089026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbec356-5bb9-43f2-814d-49480616a553 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:40.107146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee982a-18c1-472e-98a2-44534bad2283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:40.107841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee982a-18c1-472e-98a2-44534bad2283 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:40.108130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee982a-18c1-472e-98a2-44534bad2283 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:40.108409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee982a-18c1-472e-98a2-44534bad2283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:40.112176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9fa5c-11c5-47a6-90e3-712575bf153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:40.112866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9fa5c-11c5-47a6-90e3-712575bf153e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:40.113152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9fa5c-11c5-47a6-90e3-712575bf153e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:40.113418 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9fa5c-11c5-47a6-90e3-712575bf153e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:40.114154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9fa5c-11c5-47a6-90e3-712575bf153e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:40.129619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-76ee982a-18c1-472e-98a2-44534bad2283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:40.145617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc9fa5c-11c5-47a6-90e3-712575bf153e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:40.203516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 862/1717] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:40.204770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9fa5c-11c5-47a6-90e3-712575bf153e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:40.235683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc9fa5c-11c5-47a6-90e3-712575bf153e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:40.254061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70e617-b18c-4a74-9a37-d4e5e41a0312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:40.254804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70e617-b18c-4a74-9a37-d4e5e41a0312 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:40.255090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70e617-b18c-4a74-9a37-d4e5e41a0312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:40.255366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70e617-b18c-4a74-9a37-d4e5e41a0312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:40.256106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70e617-b18c-4a74-9a37-d4e5e41a0312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:40.257915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70e617-b18c-4a74-9a37-d4e5e41a0312 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:40.309053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 863/1718] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:40.310010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70e617-b18c-4a74-9a37-d4e5e41a0312 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:40.363304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70e617-b18c-4a74-9a37-d4e5e41a0312 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:40.379382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9329c-5bff-40e7-ab25-dcad1345384d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:40.379848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9329c-5bff-40e7-ab25-dcad1345384d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:40.380133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9329c-5bff-40e7-ab25-dcad1345384d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:40.380411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9329c-5bff-40e7-ab25-dcad1345384d admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:40.381464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee982a-18c1-472e-98a2-44534bad2283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:40.383041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee982a-18c1-472e-98a2-44534bad2283 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:40.388331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9329c-5bff-40e7-ab25-dcad1345384d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:40.402686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9329c-5bff-40e7-ab25-dcad1345384d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:40.436430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 864/1720] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:40.440980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 856/1720] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:40.460433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee982a-18c1-472e-98a2-44534bad2283 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:40.460629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9329c-5bff-40e7-ab25-dcad1345384d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:40.497827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9329c-5bff-40e7-ab25-dcad1345384d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:40.519947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05313998-dd72-432b-816a-c65aeb990030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:40.520909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05313998-dd72-432b-816a-c65aeb990030 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:40.521143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05313998-dd72-432b-816a-c65aeb990030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:40.521429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05313998-dd72-432b-816a-c65aeb990030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:40.522288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05313998-dd72-432b-816a-c65aeb990030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:40.524801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05313998-dd72-432b-816a-c65aeb990030 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:40.528114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee982a-18c1-472e-98a2-44534bad2283 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:40.546470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f2470-fac1-4481-a49f-0224bca1c42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:40.547895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f2470-fac1-4481-a49f-0224bca1c42f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:40.548207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f2470-fac1-4481-a49f-0224bca1c42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:40.549092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f2470-fac1-4481-a49f-0224bca1c42f admin admin] PATH_INFO: `/v3/projects/4c265ca103d04138970bf9df46b47b0d/users/0755f6f5e8c74c2aa1cb1465e9cf3af4/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:40.560781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 865/1721] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:40.561755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05313998-dd72-432b-816a-c65aeb990030 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:40.599225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05313998-dd72-432b-816a-c65aeb990030 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:40.639202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f2470-fac1-4481-a49f-0224bca1c42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c265ca103d04138970bf9df46b47b0d, user_id=0755f6f5e8c74c2aa1cb1465e9cf3af4, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:40.640499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90418b-f243-4a1a-968a-d99cc2d54fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:40.645267 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90418b-f243-4a1a-968a-d99cc2d54fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:40.645590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90418b-f243-4a1a-968a-d99cc2d54fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:40.647164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90418b-f243-4a1a-968a-d99cc2d54fb2 admin admin] PATH_INFO: `/v3/projects/7d769c9542c14140a6c9e3c6355aaa01/users/f5c48ae875074cc8ac12e0ebe179c908/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:40.651774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f2470-fac1-4481-a49f-0224bca1c42f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:40.721723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 857/1722] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:40 2022] PUT /identity/v3/projects/4c265ca103d04138970bf9df46b47b0d/users/0755f6f5e8c74c2aa1cb1465e9cf3af4/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:40.730293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d65b0-1bda-44bb-8c75-47aa282c9657 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:40.734283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d65b0-1bda-44bb-8c75-47aa282c9657 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:40.734786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d65b0-1bda-44bb-8c75-47aa282c9657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:40.742763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90418b-f243-4a1a-968a-d99cc2d54fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d769c9542c14140a6c9e3c6355aaa01, user_id=f5c48ae875074cc8ac12e0ebe179c908, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:40.755439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db90418b-f243-4a1a-968a-d99cc2d54fb2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:40.823119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-099d65b0-1bda-44bb-8c75-47aa282c9657 None None] MFA Rules not processed for user `443be2bc27514432a34d2a4c57108b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:40.824615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-099d65b0-1bda-44bb-8c75-47aa282c9657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:40.825506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 866/1723] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:40 2022] PUT /identity/v3/projects/7d769c9542c14140a6c9e3c6355aaa01/users/f5c48ae875074cc8ac12e0ebe179c908/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:40.828779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4365a-6182-4aaf-94a1-0cdfafaf6f0f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:40.834876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4365a-6182-4aaf-94a1-0cdfafaf6f0f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:40.836643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4365a-6182-4aaf-94a1-0cdfafaf6f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:40.934177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e8b4365a-6182-4aaf-94a1-0cdfafaf6f0f None None] MFA Rules not processed for user `83f3cfb67d3147d7b856158dcf32ff03`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:40.937243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e8b4365a-6182-4aaf-94a1-0cdfafaf6f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:41.055962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 858/1724] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:41.057561 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a78b73-85df-409a-a4ee-3df9e1e8fd93 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:41.061573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a78b73-85df-409a-a4ee-3df9e1e8fd93 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:41.061573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a78b73-85df-409a-a4ee-3df9e1e8fd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:41.103915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a9a78b73-85df-409a-a4ee-3df9e1e8fd93 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:41.108135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a9a78b73-85df-409a-a4ee-3df9e1e8fd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:41.139839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:41.188215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 867/1725] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:41.188215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cc7c6-e650-458b-8a27-64e26e125825 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:41.188215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cc7c6-e650-458b-8a27-64e26e125825 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:41.188215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cc7c6-e650-458b-8a27-64e26e125825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:41.228342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:41.250141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f98cc7c6-e650-458b-8a27-64e26e125825 None None] MFA Rules not processed for user `f5c48ae875074cc8ac12e0ebe179c908`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:41.254878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f98cc7c6-e650-458b-8a27-64e26e125825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:41.271324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 859/1726] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:41.274689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98045c0b-54e7-43dd-b2ef-572fa0d68205 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:41.275891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98045c0b-54e7-43dd-b2ef-572fa0d68205 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:41.276846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98045c0b-54e7-43dd-b2ef-572fa0d68205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:41.350411 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-98045c0b-54e7-43dd-b2ef-572fa0d68205 None None] MFA Rules not processed for user `443be2bc27514432a34d2a4c57108b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:41.354851 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-98045c0b-54e7-43dd-b2ef-572fa0d68205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:41.414924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 860/1727] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:41.417180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f831e24-affe-4efc-a368-e82e7a536ffe None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:41.417342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f831e24-affe-4efc-a368-e82e7a536ffe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:41.417596 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f831e24-affe-4efc-a368-e82e7a536ffe None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:41.424574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 861/1728] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:04:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:04:41.427520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f831e24-affe-4efc-a368-e82e7a536ffe None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:41.441828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1f831e24-affe-4efc-a368-e82e7a536ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:41.480643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 868/1729] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:41.481743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cc7c6-e650-458b-8a27-64e26e125825 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:41.497263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98cc7c6-e650-458b-8a27-64e26e125825 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:41.522542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecd695-b6af-4363-9ea8-4b000a4cb564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:41.523252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecd695-b6af-4363-9ea8-4b000a4cb564 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:41.523554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecd695-b6af-4363-9ea8-4b000a4cb564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:41.523836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecd695-b6af-4363-9ea8-4b000a4cb564 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:41.524578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecd695-b6af-4363-9ea8-4b000a4cb564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:41.527001 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecd695-b6af-4363-9ea8-4b000a4cb564 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:41.697486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 869/1730] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:41 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:41.704622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecd695-b6af-4363-9ea8-4b000a4cb564 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:41.907303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f831e24-affe-4efc-a368-e82e7a536ffe None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:41.945571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecd695-b6af-4363-9ea8-4b000a4cb564 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:41.965783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84607a74-565f-4c1c-a930-9d57c0919ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:41.967027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607a74-565f-4c1c-a930-9d57c0919ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:41.967027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607a74-565f-4c1c-a930-9d57c0919ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:41.967417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84607a74-565f-4c1c-a930-9d57c0919ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:41.968181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84607a74-565f-4c1c-a930-9d57c0919ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:41.970198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84607a74-565f-4c1c-a930-9d57c0919ee5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:42.029049 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 870/1731] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:41 2022] POST /identity/v3/users => generated 290 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:42.031555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84607a74-565f-4c1c-a930-9d57c0919ee5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:42.036450 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-84607a74-565f-4c1c-a930-9d57c0919ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:42.152502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed2be1-dd9d-4c27-ad69-3856c7a94c62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:42.154103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed2be1-dd9d-4c27-ad69-3856c7a94c62 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:42.154732 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed2be1-dd9d-4c27-ad69-3856c7a94c62 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:42.155027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed2be1-dd9d-4c27-ad69-3856c7a94c62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:42.231996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84607a74-565f-4c1c-a930-9d57c0919ee5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:42.238139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bddc6e-987d-4ca0-8bf3-f260df2d72d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:42.238865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bddc6e-987d-4ca0-8bf3-f260df2d72d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:42.238865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bddc6e-987d-4ca0-8bf3-f260df2d72d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:42.239455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bddc6e-987d-4ca0-8bf3-f260df2d72d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:42.268188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bddc6e-987d-4ca0-8bf3-f260df2d72d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:42.269635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bddc6e-987d-4ca0-8bf3-f260df2d72d3 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:42.294324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 871/1732] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:42.297833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fbde3-3d23-4c6f-a24f-1fff64319171 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:42.298127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fbde3-3d23-4c6f-a24f-1fff64319171 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:42.307304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fbde3-3d23-4c6f-a24f-1fff64319171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:42.360162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8c2fbde3-3d23-4c6f-a24f-1fff64319171 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:42.365340 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8c2fbde3-3d23-4c6f-a24f-1fff64319171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:42.383534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:42.395306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 872/1733] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:42.396192 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fbde3-3d23-4c6f-a24f-1fff64319171 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:42.414211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:42.425946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed2be1-dd9d-4c27-ad69-3856c7a94c62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:42.429611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed2be1-dd9d-4c27-ad69-3856c7a94c62 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:42.431567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fbde3-3d23-4c6f-a24f-1fff64319171 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:42.453534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00c918-4696-4b2e-8c7c-a43d0f4904d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:42.454179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00c918-4696-4b2e-8c7c-a43d0f4904d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:42.454495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00c918-4696-4b2e-8c7c-a43d0f4904d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:42.454802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00c918-4696-4b2e-8c7c-a43d0f4904d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:42.455547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00c918-4696-4b2e-8c7c-a43d0f4904d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:42.457428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00c918-4696-4b2e-8c7c-a43d0f4904d2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:42.494460 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 873/1734] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:42.502538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ab7e8-c65c-4341-8d98-1fff9f26f1d7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:42.502914 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ab7e8-c65c-4341-8d98-1fff9f26f1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:42.513176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ab7e8-c65c-4341-8d98-1fff9f26f1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:42.525698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 862/1735] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 1100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:42.527168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed2be1-dd9d-4c27-ad69-3856c7a94c62 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:42.568742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed2be1-dd9d-4c27-ad69-3856c7a94c62 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:42.574754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f18ab7e8-c65c-4341-8d98-1fff9f26f1d7 None None] MFA Rules not processed for user `8145168b1d02438886cd0865844f7480`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:42.576042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f18ab7e8-c65c-4341-8d98-1fff9f26f1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:42.596580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676aafb-c3cd-4304-9571-a91a059ae518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:42.597276 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676aafb-c3cd-4304-9571-a91a059ae518 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:42.597551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676aafb-c3cd-4304-9571-a91a059ae518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:42.597830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676aafb-c3cd-4304-9571-a91a059ae518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:42.598616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676aafb-c3cd-4304-9571-a91a059ae518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:42.600272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676aafb-c3cd-4304-9571-a91a059ae518 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:42.655324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 863/1736] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:42.656352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676aafb-c3cd-4304-9571-a91a059ae518 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:42.696084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676aafb-c3cd-4304-9571-a91a059ae518 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:42.711313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7317c3-6b40-4841-a156-a2f63453574b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:42.712627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7317c3-6b40-4841-a156-a2f63453574b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:42.712924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7317c3-6b40-4841-a156-a2f63453574b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:42.713318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7317c3-6b40-4841-a156-a2f63453574b admin admin] PATH_INFO: `/v3/projects/87a3b57c479f42e98efcdc9a089c3a41/users/dd3549e3cb0847668825318bc9a2ee50/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:42.753401 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7317c3-6b40-4841-a156-a2f63453574b admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a3b57c479f42e98efcdc9a089c3a41, user_id=dd3549e3cb0847668825318bc9a2ee50, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:42.756598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7317c3-6b40-4841-a156-a2f63453574b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:42.791472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 874/1737] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:42.803636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 864/1738] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:42 2022] PUT /identity/v3/projects/87a3b57c479f42e98efcdc9a089c3a41/users/dd3549e3cb0847668825318bc9a2ee50/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:42.811419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2fbed-2189-431d-b745-f959cf48db01 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:42.811962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2fbed-2189-431d-b745-f959cf48db01 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:42.812449 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2fbed-2189-431d-b745-f959cf48db01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:42.917846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1ba2fbed-2189-431d-b745-f959cf48db01 None None] MFA Rules not processed for user `dd3549e3cb0847668825318bc9a2ee50`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:42.919462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1ba2fbed-2189-431d-b745-f959cf48db01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:43.128661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 875/1739] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:43.129997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7317c3-6b40-4841-a156-a2f63453574b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:43.457490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7317c3-6b40-4841-a156-a2f63453574b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:43.465472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4386bd-2451-4370-8060-3d740afa3457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:43.466311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4386bd-2451-4370-8060-3d740afa3457 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:43.466764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4386bd-2451-4370-8060-3d740afa3457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:43.468672 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4386bd-2451-4370-8060-3d740afa3457 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:43.469433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4386bd-2451-4370-8060-3d740afa3457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:43.472847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4386bd-2451-4370-8060-3d740afa3457 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:43.487501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee293d-fdfb-4860-9b48-90ec68c86745 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:43.488043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee293d-fdfb-4860-9b48-90ec68c86745 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:43.488043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee293d-fdfb-4860-9b48-90ec68c86745 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:43.489915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 876/1740] 10.222.0.57 () {54 vars in 912 bytes} [Thu May 26 21:04:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:04:43.508895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1effb17-ff8d-409d-9fac-48c06af86715 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:43.509803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1effb17-ff8d-409d-9fac-48c06af86715 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:43.510283 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1effb17-ff8d-409d-9fac-48c06af86715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:43.529604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 865/1741] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:43 2022] POST /identity/v3/projects => generated 379 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:43.539366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4386bd-2451-4370-8060-3d740afa3457 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:43.542465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c1effb17-ff8d-409d-9fac-48c06af86715 None None] MFA Rules not processed for user `94b8d8b089264b5192a651b41bf2a205`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:43.544585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c1effb17-ff8d-409d-9fac-48c06af86715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:43.671959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 877/1742] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:04:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:04:43.719703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1effb17-ff8d-409d-9fac-48c06af86715 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:43.722747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c1effb17-ff8d-409d-9fac-48c06af86715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:43.736834 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4386bd-2451-4370-8060-3d740afa3457 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:43.740939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1effb17-ff8d-409d-9fac-48c06af86715 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:43.748396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784135a-7792-4fa5-8f4e-7da24568ac3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:43.748868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784135a-7792-4fa5-8f4e-7da24568ac3a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:43.752401 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784135a-7792-4fa5-8f4e-7da24568ac3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:43.752687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784135a-7792-4fa5-8f4e-7da24568ac3a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:43.753692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784135a-7792-4fa5-8f4e-7da24568ac3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:43.753817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202962e-a072-4d04-a017-d606b6d5405c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:43.754295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202962e-a072-4d04-a017-d606b6d5405c service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:43.754597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202962e-a072-4d04-a017-d606b6d5405c service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:43.755014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202962e-a072-4d04-a017-d606b6d5405c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:43.761457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784135a-7792-4fa5-8f4e-7da24568ac3a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:43.775847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202962e-a072-4d04-a017-d606b6d5405c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:43.779370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202962e-a072-4d04-a017-d606b6d5405c service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:43.804859 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 878/1743] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:43.815830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 866/1744] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:43 2022] POST /identity/v3/users => generated 296 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:43.820281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202962e-a072-4d04-a017-d606b6d5405c service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:43.840002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202962e-a072-4d04-a017-d606b6d5405c service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:43.853603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b80fe-59fb-4e86-89a7-dfb0e0b94a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:43.854259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b80fe-59fb-4e86-89a7-dfb0e0b94a52 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:43.854814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b80fe-59fb-4e86-89a7-dfb0e0b94a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:43.855093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b80fe-59fb-4e86-89a7-dfb0e0b94a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:43.855864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b80fe-59fb-4e86-89a7-dfb0e0b94a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:43.858367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b80fe-59fb-4e86-89a7-dfb0e0b94a52 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:43.889932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 879/1745] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:04:43.894162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784135a-7792-4fa5-8f4e-7da24568ac3a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:43.906652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784135a-7792-4fa5-8f4e-7da24568ac3a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:43.918905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4179f-1260-47c7-a522-e113dcc334f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:43.920112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4179f-1260-47c7-a522-e113dcc334f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:43.922127 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4179f-1260-47c7-a522-e113dcc334f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:43.922725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4179f-1260-47c7-a522-e113dcc334f8 admin admin] PATH_INFO: `/v3/projects/62188b94a70244f9975d34a2605a3d17/users/1131a6df6ce04b03827e2a0bde22e8c9/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:43.966805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4179f-1260-47c7-a522-e113dcc334f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62188b94a70244f9975d34a2605a3d17, user_id=1131a6df6ce04b03827e2a0bde22e8c9, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:43.969972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4179f-1260-47c7-a522-e113dcc334f8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:43.998895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 867/1746] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:43 2022] PUT /identity/v3/projects/62188b94a70244f9975d34a2605a3d17/users/1131a6df6ce04b03827e2a0bde22e8c9/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:44.006330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b80fe-59fb-4e86-89a7-dfb0e0b94a52 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:44.033685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b80fe-59fb-4e86-89a7-dfb0e0b94a52 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:44.048201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:44.300336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204a75b-e035-459a-b014-53745438fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:44.301168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a75b-e035-459a-b014-53745438fbbb admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:44.301668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a75b-e035-459a-b014-53745438fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:44.302229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204a75b-e035-459a-b014-53745438fbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:44.303125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204a75b-e035-459a-b014-53745438fbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:44.304851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204a75b-e035-459a-b014-53745438fbbb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:44.323047 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 880/1747] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:44.324086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4179f-1260-47c7-a522-e113dcc334f8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:44.333890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:44.340391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4179f-1260-47c7-a522-e113dcc334f8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:44.353936 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b2e04-f95e-402c-af50-d988ce71df16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:44.355045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b2e04-f95e-402c-af50-d988ce71df16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:44.355386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b2e04-f95e-402c-af50-d988ce71df16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:44.355623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b2e04-f95e-402c-af50-d988ce71df16 admin admin] PATH_INFO: `/v3/projects/62188b94a70244f9975d34a2605a3d17/users/1131a6df6ce04b03827e2a0bde22e8c9/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:44.368888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b2e04-f95e-402c-af50-d988ce71df16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62188b94a70244f9975d34a2605a3d17, user_id=1131a6df6ce04b03827e2a0bde22e8c9, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:44.371614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b2e04-f95e-402c-af50-d988ce71df16 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:44.405879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 868/1748] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:44 2022] PUT /identity/v3/projects/62188b94a70244f9975d34a2605a3d17/users/1131a6df6ce04b03827e2a0bde22e8c9/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:44.416523 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84a566-5c6b-4dc5-83ff-39543ec7c4fd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:44.417138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84a566-5c6b-4dc5-83ff-39543ec7c4fd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:44.418050 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84a566-5c6b-4dc5-83ff-39543ec7c4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:44.537129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6e84a566-5c6b-4dc5-83ff-39543ec7c4fd None None] MFA Rules not processed for user `1131a6df6ce04b03827e2a0bde22e8c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:44.540934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6e84a566-5c6b-4dc5-83ff-39543ec7c4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:44.778329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 881/1749] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:44.788600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b2e04-f95e-402c-af50-d988ce71df16 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:44.888569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84a566-5c6b-4dc5-83ff-39543ec7c4fd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:44.938416 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b2e04-f95e-402c-af50-d988ce71df16 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:44.946389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412451a-601e-4bdc-be9f-cde5b762116a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:44.949652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412451a-601e-4bdc-be9f-cde5b762116a service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:44.950158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412451a-601e-4bdc-be9f-cde5b762116a service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:44.950473 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412451a-601e-4bdc-be9f-cde5b762116a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:44.973097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412451a-601e-4bdc-be9f-cde5b762116a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:44.978000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412451a-601e-4bdc-be9f-cde5b762116a service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:45.019093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 869/1750] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:45.066705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412451a-601e-4bdc-be9f-cde5b762116a service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:45.068226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c412451a-601e-4bdc-be9f-cde5b762116a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:45.110011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84a566-5c6b-4dc5-83ff-39543ec7c4fd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:45.121595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35434f50-d83e-4ca4-a4bd-46b58d6f8eca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:45.122405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35434f50-d83e-4ca4-a4bd-46b58d6f8eca service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:45.122825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35434f50-d83e-4ca4-a4bd-46b58d6f8eca service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:45.123210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35434f50-d83e-4ca4-a4bd-46b58d6f8eca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:45.145044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35434f50-d83e-4ca4-a4bd-46b58d6f8eca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:45.146801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35434f50-d83e-4ca4-a4bd-46b58d6f8eca service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:45.185581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 882/1751] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:45.186538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35434f50-d83e-4ca4-a4bd-46b58d6f8eca service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:45.191392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-35434f50-d83e-4ca4-a4bd-46b58d6f8eca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:45.213570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35434f50-d83e-4ca4-a4bd-46b58d6f8eca service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:45.228153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5315b5-95f2-44e6-b106-e146c1ae2cd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:45.229191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5315b5-95f2-44e6-b106-e146c1ae2cd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:45.229627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5315b5-95f2-44e6-b106-e146c1ae2cd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:45.230422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5315b5-95f2-44e6-b106-e146c1ae2cd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:45.266547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:45.290079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412451a-601e-4bdc-be9f-cde5b762116a service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:45.310035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:45.310631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:45.310915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:45.311208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] PATH_INFO: `/v3/users/dd0d57d7bf834ef98b450530f3b3c70f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:45.313291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd0d57d7bf834ef98b450530f3b3c70f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:45.315818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:45.374338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5315b5-95f2-44e6-b106-e146c1ae2cd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:45.376375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5315b5-95f2-44e6-b106-e146c1ae2cd3 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:45.390167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 883/1752] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:45.391643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c21d0-13b1-488b-8501-c72491ab1883 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:45.392270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c21d0-13b1-488b-8501-c72491ab1883 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:45.392662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c21d0-13b1-488b-8501-c72491ab1883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:45.426155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a81c21d0-13b1-488b-8501-c72491ab1883 None None] MFA Rules not processed for user `0755f6f5e8c74c2aa1cb1465e9cf3af4`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:45.430504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a81c21d0-13b1-488b-8501-c72491ab1883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:45.492612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd0d57d7bf834ef98b450530f3b3c70f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:45.493616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd0d57d7bf834ef98b450530f3b3c70f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:45.520305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd0d57d7bf834ef98b450530f3b3c70f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:45.521318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd0d57d7bf834ef98b450530f3b3c70f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:45.564335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-09e362ce-d8de-47ff-ba45-448f99751959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd0d57d7bf834ef98b450530f3b3c70f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:45.610072 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 884/1753] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:45.615242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c21d0-13b1-488b-8501-c72491ab1883 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:45.617517 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 870/1754] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:45 2022] DELETE /identity/v3/users/dd0d57d7bf834ef98b450530f3b3c70f => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:45.618330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a81c21d0-13b1-488b-8501-c72491ab1883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:45.622094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6ba52-3f2b-4f9b-b738-c8a8419811d6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:45.622397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6ba52-3f2b-4f9b-b738-c8a8419811d6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:45.622713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6ba52-3f2b-4f9b-b738-c8a8419811d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:45.651284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c21d0-13b1-488b-8501-c72491ab1883 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:45.706703 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9fc6ba52-3f2b-4f9b-b738-c8a8419811d6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:45.711437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9fc6ba52-3f2b-4f9b-b738-c8a8419811d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:45.738128 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc468a0-a942-4a97-8ffc-66be678c7ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:45.739072 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc468a0-a942-4a97-8ffc-66be678c7ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:45.739498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc468a0-a942-4a97-8ffc-66be678c7ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:45.740194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc468a0-a942-4a97-8ffc-66be678c7ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:45.741145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc468a0-a942-4a97-8ffc-66be678c7ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:45.743559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc468a0-a942-4a97-8ffc-66be678c7ddc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:45.748089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:45.824207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 885/1755] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:45 2022] POST /identity/v3/projects => generated 405 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:04:45.828953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc468a0-a942-4a97-8ffc-66be678c7ddc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:45.938142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:46.028445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc468a0-a942-4a97-8ffc-66be678c7ddc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:46.034015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb41793-03e2-46b7-b16d-639bee654047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:46.034814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb41793-03e2-46b7-b16d-639bee654047 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:46.035104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb41793-03e2-46b7-b16d-639bee654047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:46.035401 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb41793-03e2-46b7-b16d-639bee654047 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:46.036150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb41793-03e2-46b7-b16d-639bee654047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:46.046668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb41793-03e2-46b7-b16d-639bee654047 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:46.050288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 871/1756] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:46.052826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415249f5-3880-41d8-9f96-2f1c0e236c15 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:46.056500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415249f5-3880-41d8-9f96-2f1c0e236c15 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:46.058149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415249f5-3880-41d8-9f96-2f1c0e236c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:46.106694 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 886/1757] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:45 2022] POST /identity/v3/users => generated 309 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:46.108801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb41793-03e2-46b7-b16d-639bee654047 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:46.113900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5eb41793-03e2-46b7-b16d-639bee654047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:46.129917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-415249f5-3880-41d8-9f96-2f1c0e236c15 None None] MFA Rules not processed for user `443be2bc27514432a34d2a4c57108b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:46.136246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-415249f5-3880-41d8-9f96-2f1c0e236c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:46.226687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb41793-03e2-46b7-b16d-639bee654047 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:46.247532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95889038-5f5c-4636-9e9a-060b91ff354d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:46.248475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95889038-5f5c-4636-9e9a-060b91ff354d service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:46.249202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95889038-5f5c-4636-9e9a-060b91ff354d service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:46.250551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95889038-5f5c-4636-9e9a-060b91ff354d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:46.283456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95889038-5f5c-4636-9e9a-060b91ff354d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:46.284992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95889038-5f5c-4636-9e9a-060b91ff354d service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:46.334691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 887/1758] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:46.335714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95889038-5f5c-4636-9e9a-060b91ff354d service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:46.366969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95889038-5f5c-4636-9e9a-060b91ff354d service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:46.378727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a1247-4599-4b07-bcb6-bb0c716fd861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:46.379981 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a1247-4599-4b07-bcb6-bb0c716fd861 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:46.380527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a1247-4599-4b07-bcb6-bb0c716fd861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:46.380947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a1247-4599-4b07-bcb6-bb0c716fd861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:46.381813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a1247-4599-4b07-bcb6-bb0c716fd861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:46.385749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a1247-4599-4b07-bcb6-bb0c716fd861 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:46.400032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 872/1759] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:46.410379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 888/1760] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:46.414961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415249f5-3880-41d8-9f96-2f1c0e236c15 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:46.444673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415249f5-3880-41d8-9f96-2f1c0e236c15 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:46.452188 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e54b3-9601-457f-acd9-d04823106e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:46.453281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e54b3-9601-457f-acd9-d04823106e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:46.453720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e54b3-9601-457f-acd9-d04823106e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:46.454008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e54b3-9601-457f-acd9-d04823106e52 admin admin] PATH_INFO: `/v3/projects/25261eb9e0fc4f8c806b22d78981b934/users/564a2ad8fc1f4cb5b455851086f56b3f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:46.479074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e54b3-9601-457f-acd9-d04823106e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25261eb9e0fc4f8c806b22d78981b934, user_id=564a2ad8fc1f4cb5b455851086f56b3f, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:46.482244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562e54b3-9601-457f-acd9-d04823106e52 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:46.508952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 873/1761] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:46 2022] PUT /identity/v3/projects/25261eb9e0fc4f8c806b22d78981b934/users/564a2ad8fc1f4cb5b455851086f56b3f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:46.512793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a1247-4599-4b07-bcb6-bb0c716fd861 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:46.533767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a1247-4599-4b07-bcb6-bb0c716fd861 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:46.710880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fad1fc-f460-402e-884e-57c418b9a323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:46.712223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fad1fc-f460-402e-884e-57c418b9a323 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:46.714430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fad1fc-f460-402e-884e-57c418b9a323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:46.719364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fad1fc-f460-402e-884e-57c418b9a323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:46.724777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fad1fc-f460-402e-884e-57c418b9a323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:46.733582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fad1fc-f460-402e-884e-57c418b9a323 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:46.769624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 889/1762] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:46.773804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e54b3-9601-457f-acd9-d04823106e52 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:46.790688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e54b3-9601-457f-acd9-d04823106e52 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:46.797877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c62e73-fbcb-446a-a9cf-3784fd64d5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:46.817495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62e73-fbcb-446a-a9cf-3784fd64d5bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:46.818952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62e73-fbcb-446a-a9cf-3784fd64d5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:46.820231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c62e73-fbcb-446a-a9cf-3784fd64d5bb admin admin] PATH_INFO: `/v3/projects/25261eb9e0fc4f8c806b22d78981b934/users/564a2ad8fc1f4cb5b455851086f56b3f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:46.825528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c62e73-fbcb-446a-a9cf-3784fd64d5bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=25261eb9e0fc4f8c806b22d78981b934, user_id=564a2ad8fc1f4cb5b455851086f56b3f, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:46.844266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c62e73-fbcb-446a-a9cf-3784fd64d5bb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:46.869789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fad1fc-f460-402e-884e-57c418b9a323 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:46.871686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-10fad1fc-f460-402e-884e-57c418b9a323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:46.892719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 874/1763] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:46 2022] PUT /identity/v3/projects/25261eb9e0fc4f8c806b22d78981b934/users/564a2ad8fc1f4cb5b455851086f56b3f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:46.896272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fad1fc-f460-402e-884e-57c418b9a323 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:47.055219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:47.081495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:47.082392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:47.082760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:47.083068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] PATH_INFO: `/v3/projects/73887c3d2f5a4a2793387257395c44d8` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:47.087103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] RBAC: Authorizing `identity:delete_project(project_id=73887c3d2f5a4a2793387257395c44d8)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:47.089053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:47.234091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73887c3d2f5a4a2793387257395c44d8', 'request_id': 'req-4c8dd22e-9076-4694-ae89-a36f2226522d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:47.251329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73887c3d2f5a4a2793387257395c44d8', 'request_id': 'req-4c8dd22e-9076-4694-ae89-a36f2226522d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:47.269156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73887c3d2f5a4a2793387257395c44d8', 'request_id': 'req-4c8dd22e-9076-4694-ae89-a36f2226522d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:47.292777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] The token cache is being invalidate because project 73887c3d2f5a4a2793387257395c44d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:47.292929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:47.293756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:47.296297 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 890/1764] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:46 2022] DELETE /identity/v3/projects/73887c3d2f5a4a2793387257395c44d8 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:47.300895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c62e73-fbcb-446a-a9cf-3784fd64d5bb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:47.303327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-32c62e73-fbcb-446a-a9cf-3784fd64d5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:47.332589 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c62e73-fbcb-446a-a9cf-3784fd64d5bb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:47.515121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:47.516291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:47.516809 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:47.518064 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] PATH_INFO: `/v3/users/7a7c2ae9eb17496fb038328f66e86ffe` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:47.519794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a7c2ae9eb17496fb038328f66e86ffe)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:47.523328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:47.557888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:47.639730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a7c2ae9eb17496fb038328f66e86ffe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:47.640456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a7c2ae9eb17496fb038328f66e86ffe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:47.660368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a7c2ae9eb17496fb038328f66e86ffe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:47.661093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a7c2ae9eb17496fb038328f66e86ffe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:47.698410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2371770-075a-4f79-83f9-cc789a5a2777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a7c2ae9eb17496fb038328f66e86ffe'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:47.737785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 875/1765] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:47 2022] DELETE /identity/v3/users/7a7c2ae9eb17496fb038328f66e86ffe => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:48.614854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:48.633791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8dd22e-9076-4694-ae89-a36f2226522d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:48.871730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:48.872748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:48.873036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:48.873357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] PATH_INFO: `/v3/projects/a82ff6593d3e4905b70d250bc3dfc60c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:48.886860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a82ff6593d3e4905b70d250bc3dfc60c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:48.889448 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:49.069770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a82ff6593d3e4905b70d250bc3dfc60c', 'request_id': 'req-1e0834c4-f3be-4b7e-816c-84be9317d9d0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:49.087976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a82ff6593d3e4905b70d250bc3dfc60c', 'request_id': 'req-1e0834c4-f3be-4b7e-816c-84be9317d9d0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:49.099730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a82ff6593d3e4905b70d250bc3dfc60c', 'request_id': 'req-1e0834c4-f3be-4b7e-816c-84be9317d9d0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:49.179668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] The token cache is being invalidate because project a82ff6593d3e4905b70d250bc3dfc60c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:49.180009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:49.183737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e0834c4-f3be-4b7e-816c-84be9317d9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:49.191794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 891/1766] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:48 2022] DELETE /identity/v3/projects/a82ff6593d3e4905b70d250bc3dfc60c => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:49.196599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d053f-a462-4fc1-be91-70ca30830631 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:49.197018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d053f-a462-4fc1-be91-70ca30830631 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:49.197137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d053f-a462-4fc1-be91-70ca30830631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:49.217312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f4261-bb2a-42a4-a1ab-9eac428004b1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:49.217745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f4261-bb2a-42a4-a1ab-9eac428004b1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:49.220783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f4261-bb2a-42a4-a1ab-9eac428004b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:49.241740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3b1d053f-a462-4fc1-be91-70ca30830631 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:49.243022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3b1d053f-a462-4fc1-be91-70ca30830631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:49.326944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:49.365347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6c1f4261-bb2a-42a4-a1ab-9eac428004b1 None None] MFA Rules not processed for user `564a2ad8fc1f4cb5b455851086f56b3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:49.369799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6c1f4261-bb2a-42a4-a1ab-9eac428004b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:49.447080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 876/1767] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:49.455683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58cd85-4b6c-49ee-abb0-7eb70b96f042 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:49.456032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58cd85-4b6c-49ee-abb0-7eb70b96f042 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:49.456395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58cd85-4b6c-49ee-abb0-7eb70b96f042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:49.510647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9b58cd85-4b6c-49ee-abb0-7eb70b96f042 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:49.512008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9b58cd85-4b6c-49ee-abb0-7eb70b96f042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:49.551722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 877/1768] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:49.554838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58cd85-4b6c-49ee-abb0-7eb70b96f042 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:49.579421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 892/1769] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:49 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:49.618356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58cd85-4b6c-49ee-abb0-7eb70b96f042 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:49.647361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df600384-bc21-4499-820e-7c008130d97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:49.653254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df600384-bc21-4499-820e-7c008130d97f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:49.653925 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df600384-bc21-4499-820e-7c008130d97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:49.654908 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df600384-bc21-4499-820e-7c008130d97f admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:49.655706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df600384-bc21-4499-820e-7c008130d97f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:49.660284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df600384-bc21-4499-820e-7c008130d97f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:49.665871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df600384-bc21-4499-820e-7c008130d97f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:49.716861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 878/1770] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:49.722840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f4261-bb2a-42a4-a1ab-9eac428004b1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:49.757387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f4261-bb2a-42a4-a1ab-9eac428004b1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:49.772961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c1c74-c4ab-4f81-b65b-933fd939bd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:49.774634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c1c74-c4ab-4f81-b65b-933fd939bd0b admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:49.775081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c1c74-c4ab-4f81-b65b-933fd939bd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:49.775952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c1c74-c4ab-4f81-b65b-933fd939bd0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:49.776717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c1c74-c4ab-4f81-b65b-933fd939bd0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:49.786590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0c1c74-c4ab-4f81-b65b-933fd939bd0b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:49.921903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 893/1771] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:49 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:49.926650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df600384-bc21-4499-820e-7c008130d97f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:49.937853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df600384-bc21-4499-820e-7c008130d97f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:50.072518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf55951-1619-492c-9d08-c9d7e7a46253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:50.073313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf55951-1619-492c-9d08-c9d7e7a46253 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:50.075768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf55951-1619-492c-9d08-c9d7e7a46253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:50.076288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf55951-1619-492c-9d08-c9d7e7a46253 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:50.078589 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf55951-1619-492c-9d08-c9d7e7a46253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:50.087010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf55951-1619-492c-9d08-c9d7e7a46253 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:50.133365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 879/1772] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:49 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:50.138750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c1c74-c4ab-4f81-b65b-933fd939bd0b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:50.160213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c1c74-c4ab-4f81-b65b-933fd939bd0b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:50.171073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7b60f-e1e7-4262-b6b7-f19599e9a3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:50.173623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7b60f-e1e7-4262-b6b7-f19599e9a3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:50.175639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7b60f-e1e7-4262-b6b7-f19599e9a3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:50.176298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7b60f-e1e7-4262-b6b7-f19599e9a3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:50.177078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7b60f-e1e7-4262-b6b7-f19599e9a3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:50.180824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7b60f-e1e7-4262-b6b7-f19599e9a3a9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:50.216848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf55951-1619-492c-9d08-c9d7e7a46253 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:50.217810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 894/1773] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:50.243103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf55951-1619-492c-9d08-c9d7e7a46253 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:50.261045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4d208-e23a-497c-bc21-19ff49e1b8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:50.262229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4d208-e23a-497c-bc21-19ff49e1b8be admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:50.262762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4d208-e23a-497c-bc21-19ff49e1b8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:50.263502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4d208-e23a-497c-bc21-19ff49e1b8be admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:50.264444 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4d208-e23a-497c-bc21-19ff49e1b8be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:50.268846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4d208-e23a-497c-bc21-19ff49e1b8be admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:50.281792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:50.302543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 880/1774] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:50.305420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7b60f-e1e7-4262-b6b7-f19599e9a3a9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:50.332246 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7b60f-e1e7-4262-b6b7-f19599e9a3a9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:50.349710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbeea85-54e7-4fc5-9dc9-64f269172f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:50.352071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbeea85-54e7-4fc5-9dc9-64f269172f7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:50.353660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbeea85-54e7-4fc5-9dc9-64f269172f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:50.353660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbeea85-54e7-4fc5-9dc9-64f269172f7c admin admin] PATH_INFO: `/v3/projects/615eb581cb2243c4af0a9961c3e9aef4/users/fcd43e5cbe0346b781acffe230fc63d4/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:50.408800 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbeea85-54e7-4fc5-9dc9-64f269172f7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=615eb581cb2243c4af0a9961c3e9aef4, user_id=fcd43e5cbe0346b781acffe230fc63d4, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:50.412976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbeea85-54e7-4fc5-9dc9-64f269172f7c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:50.449793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e85b70-d467-4246-92ae-ef893256dba0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:50.450368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e85b70-d467-4246-92ae-ef893256dba0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:50.451086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e85b70-d467-4246-92ae-ef893256dba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:50.453197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 895/1775] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:50 2022] PUT /identity/v3/projects/615eb581cb2243c4af0a9961c3e9aef4/users/fcd43e5cbe0346b781acffe230fc63d4/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:50.561396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-99e85b70-d467-4246-92ae-ef893256dba0 None None] MFA Rules not processed for user `fcd43e5cbe0346b781acffe230fc63d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:50.564680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-99e85b70-d467-4246-92ae-ef893256dba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:50.716369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbeea85-54e7-4fc5-9dc9-64f269172f7c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:50.719125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 881/1776] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:50.875933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbeea85-54e7-4fc5-9dc9-64f269172f7c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:50.904782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3c841-501a-4523-a7e8-314978bed293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:50.905479 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3c841-501a-4523-a7e8-314978bed293 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:50.905763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3c841-501a-4523-a7e8-314978bed293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:50.906124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3c841-501a-4523-a7e8-314978bed293 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:50.906882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3c841-501a-4523-a7e8-314978bed293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:50.909263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3c841-501a-4523-a7e8-314978bed293 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:50.960631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 896/1777] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:50 2022] POST /identity/v3/projects => generated 387 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:50.965405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e85b70-d467-4246-92ae-ef893256dba0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:50.992114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3c841-501a-4523-a7e8-314978bed293 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:50.992493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e85b70-d467-4246-92ae-ef893256dba0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:50.994011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4ad3c841-501a-4523-a7e8-314978bed293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:51.092743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:51.121858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1f332-60ee-4892-8d4f-7e02c5850e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:51.122833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1f332-60ee-4892-8d4f-7e02c5850e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:51.123266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1f332-60ee-4892-8d4f-7e02c5850e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:51.123723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1f332-60ee-4892-8d4f-7e02c5850e14 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:51.124804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1f332-60ee-4892-8d4f-7e02c5850e14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:51.127798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1f332-60ee-4892-8d4f-7e02c5850e14 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:51.156842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 882/1778] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:50 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:51.162838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1f332-60ee-4892-8d4f-7e02c5850e14 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:51.175077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3c841-501a-4523-a7e8-314978bed293 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:51.181573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:51.183902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:51.184429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:51.184787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] PATH_INFO: `/v3/users/f5c48ae875074cc8ac12e0ebe179c908` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:51.186478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c48ae875074cc8ac12e0ebe179c908)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:51.188791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:51.193760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1f332-60ee-4892-8d4f-7e02c5850e14 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:51.205441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca09f8-b329-459d-8290-22ac8b565b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:51.207625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca09f8-b329-459d-8290-22ac8b565b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:51.208106 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca09f8-b329-459d-8290-22ac8b565b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:51.208606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca09f8-b329-459d-8290-22ac8b565b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:51.210471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca09f8-b329-459d-8290-22ac8b565b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:51.214185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca09f8-b329-459d-8290-22ac8b565b96 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:51.233324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 883/1779] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:51.237076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca09f8-b329-459d-8290-22ac8b565b96 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:51.252559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca09f8-b329-459d-8290-22ac8b565b96 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:51.258805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c48ae875074cc8ac12e0ebe179c908'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:51.259775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578b8a2-83a9-4c31-b302-1e549ae5f2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:51.261790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578b8a2-83a9-4c31-b302-1e549ae5f2f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:51.262384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578b8a2-83a9-4c31-b302-1e549ae5f2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:51.262830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578b8a2-83a9-4c31-b302-1e549ae5f2f0 admin admin] PATH_INFO: `/v3/projects/344f63da597c4f039723a694338d8885/users/8b821f4d6f234841bec1c8e547034c3f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:51.293760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c48ae875074cc8ac12e0ebe179c908'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:51.306422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578b8a2-83a9-4c31-b302-1e549ae5f2f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344f63da597c4f039723a694338d8885, user_id=8b821f4d6f234841bec1c8e547034c3f, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:51.309803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578b8a2-83a9-4c31-b302-1e549ae5f2f0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:51.314026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c48ae875074cc8ac12e0ebe179c908'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:51.315031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c48ae875074cc8ac12e0ebe179c908'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:51.325666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3c64ddb3-b2ac-47a0-97b6-118cc8b4a5b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c48ae875074cc8ac12e0ebe179c908'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:51.334188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 897/1780] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:50 2022] DELETE /identity/v3/users/f5c48ae875074cc8ac12e0ebe179c908 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:51.339377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 884/1781] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:51 2022] PUT /identity/v3/projects/344f63da597c4f039723a694338d8885/users/8b821f4d6f234841bec1c8e547034c3f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:51.346665 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578b8a2-83a9-4c31-b302-1e549ae5f2f0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:51.347175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b54d95-10d6-43dd-a86d-fc0cb76d5a36 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:51.347567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b54d95-10d6-43dd-a86d-fc0cb76d5a36 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:51.348094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b54d95-10d6-43dd-a86d-fc0cb76d5a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:51.348682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9578b8a2-83a9-4c31-b302-1e549ae5f2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:51.368816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d1b54d95-10d6-43dd-a86d-fc0cb76d5a36 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:51.370245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d1b54d95-10d6-43dd-a86d-fc0cb76d5a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:51.444442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:51.496858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 898/1782] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:51.526800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578b8a2-83a9-4c31-b302-1e549ae5f2f0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:51.532497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e6f70-c800-4ee2-a26d-536eb299ac25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:51.533874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e6f70-c800-4ee2-a26d-536eb299ac25 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:51.534166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e6f70-c800-4ee2-a26d-536eb299ac25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:51.534646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e6f70-c800-4ee2-a26d-536eb299ac25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:51.535388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e6f70-c800-4ee2-a26d-536eb299ac25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:51.537392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e6f70-c800-4ee2-a26d-536eb299ac25 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:51.562617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 885/1783] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:51.563923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b54d95-10d6-43dd-a86d-fc0cb76d5a36 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:51.578517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b54d95-10d6-43dd-a86d-fc0cb76d5a36 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:51.578909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e6f70-c800-4ee2-a26d-536eb299ac25 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:51.581371 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-128e6f70-c800-4ee2-a26d-536eb299ac25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:51.596655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e774a1-996b-4b6e-b3a6-9dd87899ff61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:51.597573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e774a1-996b-4b6e-b3a6-9dd87899ff61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:51.597863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e774a1-996b-4b6e-b3a6-9dd87899ff61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:51.598144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e774a1-996b-4b6e-b3a6-9dd87899ff61 admin admin] PATH_INFO: `/v3/projects/344f63da597c4f039723a694338d8885/users/8b821f4d6f234841bec1c8e547034c3f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:51.605897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e774a1-996b-4b6e-b3a6-9dd87899ff61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344f63da597c4f039723a694338d8885, user_id=8b821f4d6f234841bec1c8e547034c3f, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:51.617606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e774a1-996b-4b6e-b3a6-9dd87899ff61 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:51.642870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 899/1784] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:51 2022] PUT /identity/v3/projects/344f63da597c4f039723a694338d8885/users/8b821f4d6f234841bec1c8e547034c3f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:51.654289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b42b5-1734-4cb4-8fd1-baca04020374 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:51.654649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b42b5-1734-4cb4-8fd1-baca04020374 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:51.654999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b42b5-1734-4cb4-8fd1-baca04020374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:51.716311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e6f70-c800-4ee2-a26d-536eb299ac25 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:51.730560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-800b42b5-1734-4cb4-8fd1-baca04020374 None None] MFA Rules not processed for user `8b821f4d6f234841bec1c8e547034c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:51.732207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-800b42b5-1734-4cb4-8fd1-baca04020374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:51.762024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:51.855984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 900/1785] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:04:51.856192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9feed7-a02e-4d4a-a96e-223681dd101d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:51.856877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9feed7-a02e-4d4a-a96e-223681dd101d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:51.857209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9feed7-a02e-4d4a-a96e-223681dd101d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:51.857391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9feed7-a02e-4d4a-a96e-223681dd101d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:51.869309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b42b5-1734-4cb4-8fd1-baca04020374 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:51.872689 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-800b42b5-1734-4cb4-8fd1-baca04020374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:51.972349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9feed7-a02e-4d4a-a96e-223681dd101d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:51.976046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9feed7-a02e-4d4a-a96e-223681dd101d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:52.013787 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 886/1786] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:52.021294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b42b5-1734-4cb4-8fd1-baca04020374 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:52.028168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6b95e-5f53-47a8-89eb-919bcc8b9aed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:52.030162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb84e7-95dd-45b3-8104-326ea192028f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:52.030475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb84e7-95dd-45b3-8104-326ea192028f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:52.031254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb84e7-95dd-45b3-8104-326ea192028f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:52.033155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 887/1787] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:04:52.035144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6b95e-5f53-47a8-89eb-919bcc8b9aed service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:52.035831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6b95e-5f53-47a8-89eb-919bcc8b9aed service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:52.035938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6b95e-5f53-47a8-89eb-919bcc8b9aed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:52.050293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6b95e-5f53-47a8-89eb-919bcc8b9aed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:52.052156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6b95e-5f53-47a8-89eb-919bcc8b9aed service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:52.067998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01974fee-2f39-4092-86f5-a38d74ffb87c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:52.068306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01974fee-2f39-4092-86f5-a38d74ffb87c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:52.068594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01974fee-2f39-4092-86f5-a38d74ffb87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:52.087400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 901/1788] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:52.110042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-01974fee-2f39-4092-86f5-a38d74ffb87c None None] MFA Rules not processed for user `4edb53dc871047ce89ebb3b90143a6de`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:52.112336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-01974fee-2f39-4092-86f5-a38d74ffb87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:52.120404 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6b95e-5f53-47a8-89eb-919bcc8b9aed service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:52.137867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6b95e-5f53-47a8-89eb-919bcc8b9aed service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:52.151757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:52.153785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:52.153903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:52.156298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] PATH_INFO: `/v3/users/fcd43e5cbe0346b781acffe230fc63d4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:52.161058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcd43e5cbe0346b781acffe230fc63d4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:52.163854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:52.236852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:52.245764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcd43e5cbe0346b781acffe230fc63d4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:52.285004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 888/1789] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:04:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:52.295599 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcd43e5cbe0346b781acffe230fc63d4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:52.316689 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcd43e5cbe0346b781acffe230fc63d4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:52.317484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcd43e5cbe0346b781acffe230fc63d4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:52.332015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcd43e5cbe0346b781acffe230fc63d4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:52.339156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 902/1790] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:52 2022] DELETE /identity/v3/users/fcd43e5cbe0346b781acffe230fc63d4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:52.347462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842cd82-d619-4b69-9b76-ab48da3f6d06 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:52.347880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842cd82-d619-4b69-9b76-ab48da3f6d06 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:52.348243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842cd82-d619-4b69-9b76-ab48da3f6d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:52.359836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:52.361959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:52.384581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5842cd82-d619-4b69-9b76-ab48da3f6d06 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:52.385924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5842cd82-d619-4b69-9b76-ab48da3f6d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:52.453340 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:52.508307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 889/1791] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:52.528291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842cd82-d619-4b69-9b76-ab48da3f6d06 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:52.531046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5842cd82-d619-4b69-9b76-ab48da3f6d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:52.539854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06a24a-624f-4435-a86e-54d3cb21fec0 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:52.559303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b20926-2108-4af1-8617-72276af1f6dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:52.560422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b20926-2108-4af1-8617-72276af1f6dc service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:52.564473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b20926-2108-4af1-8617-72276af1f6dc service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:52.567453 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b20926-2108-4af1-8617-72276af1f6dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:52.573592 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-59b20926-2108-4af1-8617-72276af1f6dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:52.604481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b20926-2108-4af1-8617-72276af1f6dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:52.604825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b20926-2108-4af1-8617-72276af1f6dc service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:52.627749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 903/1792] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:52.628325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7ff8b-0270-4238-b114-79701a05e7f5 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:52.628624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7ff8b-0270-4238-b114-79701a05e7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:52.628907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7ff8b-0270-4238-b114-79701a05e7f5 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:52.630836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 904/1793] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:04:52.656769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842cd82-d619-4b69-9b76-ab48da3f6d06 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:52.662378 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef26c2-5c3d-4dc7-8021-32e8e090e58d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:52.663341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef26c2-5c3d-4dc7-8021-32e8e090e58d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:52.663896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef26c2-5c3d-4dc7-8021-32e8e090e58d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:52.664345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef26c2-5c3d-4dc7-8021-32e8e090e58d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:52.679844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef26c2-5c3d-4dc7-8021-32e8e090e58d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:52.681970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef26c2-5c3d-4dc7-8021-32e8e090e58d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:52.706079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 890/1794] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:04:52.717499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952d443-59fe-43b1-b863-67ec6354c4ab None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:52.717791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952d443-59fe-43b1-b863-67ec6354c4ab None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:52.718720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952d443-59fe-43b1-b863-67ec6354c4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:52.745275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e952d443-59fe-43b1-b863-67ec6354c4ab None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:52.746517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e952d443-59fe-43b1-b863-67ec6354c4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:52.836313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 905/1795] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:04:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:52.851383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef26c2-5c3d-4dc7-8021-32e8e090e58d service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:52.853196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-16ef26c2-5c3d-4dc7-8021-32e8e090e58d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:52.876824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef26c2-5c3d-4dc7-8021-32e8e090e58d service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:52.887563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be56f3-ddf3-4438-9524-78755d9cafb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:52.888416 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be56f3-ddf3-4438-9524-78755d9cafb4 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:52.888716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be56f3-ddf3-4438-9524-78755d9cafb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:52.889776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be56f3-ddf3-4438-9524-78755d9cafb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:52.902959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be56f3-ddf3-4438-9524-78755d9cafb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:52.905208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be56f3-ddf3-4438-9524-78755d9cafb4 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:52.917281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 891/1796] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:04:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:04:53.326721 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952d443-59fe-43b1-b863-67ec6354c4ab None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:53.330094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e952d443-59fe-43b1-b863-67ec6354c4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:53.355356 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952d443-59fe-43b1-b863-67ec6354c4ab None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:53.365942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:53.366984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:53.367343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:53.367695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] PATH_INFO: `/v3/users/0755f6f5e8c74c2aa1cb1465e9cf3af4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:53.369173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0755f6f5e8c74c2aa1cb1465e9cf3af4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:53.372934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:53.474597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0755f6f5e8c74c2aa1cb1465e9cf3af4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.498593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be56f3-ddf3-4438-9524-78755d9cafb4 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:53.500346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-76be56f3-ddf3-4438-9524-78755d9cafb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:53.519829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be56f3-ddf3-4438-9524-78755d9cafb4 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:53.521732 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0755f6f5e8c74c2aa1cb1465e9cf3af4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.525181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:53.526521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:53.527066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:53.527441 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] PATH_INFO: `/v3/projects/7d769c9542c14140a6c9e3c6355aaa01` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:53.529684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d769c9542c14140a6c9e3c6355aaa01)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:53.533724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:53.540389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0755f6f5e8c74c2aa1cb1465e9cf3af4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.544348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0755f6f5e8c74c2aa1cb1465e9cf3af4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.566177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0755f6f5e8c74c2aa1cb1465e9cf3af4'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.582103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 906/1797] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:53 2022] DELETE /identity/v3/users/0755f6f5e8c74c2aa1cb1465e9cf3af4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:53.610308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:53.615573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:53.637143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d769c9542c14140a6c9e3c6355aaa01', 'request_id': 'req-3bb42811-9241-44eb-af8e-e56a308c73af'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.644990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d769c9542c14140a6c9e3c6355aaa01', 'request_id': 'req-3bb42811-9241-44eb-af8e-e56a308c73af'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.654329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d769c9542c14140a6c9e3c6355aaa01', 'request_id': 'req-3bb42811-9241-44eb-af8e-e56a308c73af'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.662498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:53.694056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:53.696440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] The token cache is being invalidate because project 7d769c9542c14140a6c9e3c6355aaa01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:53.696720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.697572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.704199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 892/1798] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:53 2022] DELETE /identity/v3/projects/7d769c9542c14140a6c9e3c6355aaa01 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:53.706210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:53.709630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:53.756098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1f56-409a-4a58-8c3d-05900860c313 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:53.839879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:53.840785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:53.841077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:53.841359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] PATH_INFO: `/v3/projects/615eb581cb2243c4af0a9961c3e9aef4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:53.843019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615eb581cb2243c4af0a9961c3e9aef4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:53.849103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:53.912435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb42811-9241-44eb-af8e-e56a308c73af admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:53.920050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:53.921354 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:53.921750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:53.922102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] PATH_INFO: `/v3/users/dd3549e3cb0847668825318bc9a2ee50` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:53.924317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd3549e3cb0847668825318bc9a2ee50)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:53.927664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:53.953714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615eb581cb2243c4af0a9961c3e9aef4', 'request_id': 'req-2f400411-16a7-47ea-aa35-a707bca92e96'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.961632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615eb581cb2243c4af0a9961c3e9aef4', 'request_id': 'req-2f400411-16a7-47ea-aa35-a707bca92e96'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.970390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615eb581cb2243c4af0a9961c3e9aef4', 'request_id': 'req-2f400411-16a7-47ea-aa35-a707bca92e96'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.986563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] The token cache is being invalidate because project 615eb581cb2243c4af0a9961c3e9aef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:53.986995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.987749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.989844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 907/1799] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:53 2022] DELETE /identity/v3/projects/615eb581cb2243c4af0a9961c3e9aef4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:53.990697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:53.993957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd3549e3cb0847668825318bc9a2ee50'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:53.994254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:53.995219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd3549e3cb0847668825318bc9a2ee50'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.007070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd3549e3cb0847668825318bc9a2ee50'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.007898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd3549e3cb0847668825318bc9a2ee50'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.026878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd3549e3cb0847668825318bc9a2ee50'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.060922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 893/1800] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:53 2022] DELETE /identity/v3/users/dd3549e3cb0847668825318bc9a2ee50 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:54.074229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:54.086945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:54.098977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f400411-16a7-47ea-aa35-a707bca92e96 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:54.105416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:54.194525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:54.195373 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:54.195657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:54.196159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] PATH_INFO: `/v3/projects/4c265ca103d04138970bf9df46b47b0d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:54.198184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c265ca103d04138970bf9df46b47b0d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:54.201325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:54.283340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9e875-bdec-4bc7-bf1b-7ae262af818b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:54.285480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c265ca103d04138970bf9df46b47b0d', 'request_id': 'req-bceaa682-48fa-44fa-91aa-134611cfb711'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.294972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c265ca103d04138970bf9df46b47b0d', 'request_id': 'req-bceaa682-48fa-44fa-91aa-134611cfb711'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.302468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c265ca103d04138970bf9df46b47b0d', 'request_id': 'req-bceaa682-48fa-44fa-91aa-134611cfb711'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.310176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:54.322467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] The token cache is being invalidate because project 4c265ca103d04138970bf9df46b47b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:54.323776 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.325290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.328844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 908/1801] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:53 2022] DELETE /identity/v3/projects/4c265ca103d04138970bf9df46b47b0d => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:54.333257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:54.337541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:54.389647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:54.390642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:54.393553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:54.393869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] PATH_INFO: `/v3/users/8b821f4d6f234841bec1c8e547034c3f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:54.397077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b821f4d6f234841bec1c8e547034c3f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:54.399826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:54.445764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceaa682-48fa-44fa-91aa-134611cfb711 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:54.456543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:54.457630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:54.457974 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:54.468909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] PATH_INFO: `/v3/users/564a2ad8fc1f4cb5b455851086f56b3f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:54.473281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] RBAC: Authorizing `identity:delete_user(user_id=564a2ad8fc1f4cb5b455851086f56b3f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:54.480058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:54.489037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b821f4d6f234841bec1c8e547034c3f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.491189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b821f4d6f234841bec1c8e547034c3f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.507831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b821f4d6f234841bec1c8e547034c3f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.508563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b821f4d6f234841bec1c8e547034c3f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.544989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b821f4d6f234841bec1c8e547034c3f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.589351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 894/1802] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:54 2022] DELETE /identity/v3/users/8b821f4d6f234841bec1c8e547034c3f => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:54.606087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:54.616915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564a2ad8fc1f4cb5b455851086f56b3f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.638602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564a2ad8fc1f4cb5b455851086f56b3f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.655563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564a2ad8fc1f4cb5b455851086f56b3f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.658185 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564a2ad8fc1f4cb5b455851086f56b3f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.670166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564a2ad8fc1f4cb5b455851086f56b3f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:54.678963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 909/1803] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:54 2022] DELETE /identity/v3/users/564a2ad8fc1f4cb5b455851086f56b3f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:55.034212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:55.035984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:55.156616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad2f5f-8f73-4bcf-a594-4ba3ae9bd3b9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:55.227199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:55.242687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:55.243565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:55.244512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:55.244969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] PATH_INFO: `/v3/projects/87a3b57c479f42e98efcdc9a089c3a41` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:55.246480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a3b57c479f42e98efcdc9a089c3a41)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:55.248586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:55.298042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a3b57c479f42e98efcdc9a089c3a41', 'request_id': 'req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:55.304426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a3b57c479f42e98efcdc9a089c3a41', 'request_id': 'req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:55.310709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a3b57c479f42e98efcdc9a089c3a41', 'request_id': 'req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:55.324319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] The token cache is being invalidate because project 87a3b57c479f42e98efcdc9a089c3a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:55.324615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:55.325239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:55.327083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 895/1804] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:55 2022] DELETE /identity/v3/projects/87a3b57c479f42e98efcdc9a089c3a41 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:04:55.331994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:55.333379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:55.347348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315a3a8-92b5-41de-95b3-a5e0f1e7d48a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:55.412881 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:55.413788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:55.414077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:55.414358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] PATH_INFO: `/v3/users/1131a6df6ce04b03827e2a0bde22e8c9` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:55.415704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1131a6df6ce04b03827e2a0bde22e8c9)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:55.417710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:55.479738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1131a6df6ce04b03827e2a0bde22e8c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:55.497284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1131a6df6ce04b03827e2a0bde22e8c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:55.512058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1131a6df6ce04b03827e2a0bde22e8c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:55.513043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1131a6df6ce04b03827e2a0bde22e8c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:55.523383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1131a6df6ce04b03827e2a0bde22e8c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:55.533725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 910/1805] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:04:55 2022] DELETE /identity/v3/users/1131a6df6ce04b03827e2a0bde22e8c9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:55.936160 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:55.939542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:55.968221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cc74c-8358-490d-b1f9-9e78f883ee8c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:55.986968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:55.988482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:56.080898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:56.082080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:56.082919 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:56.083409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] PATH_INFO: `/v3/projects/62188b94a70244f9975d34a2605a3d17` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:56.084986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62188b94a70244f9975d34a2605a3d17)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:56.087242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:56.112074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:56.174651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772a7dc4-a49d-4422-882b-afa685984d1b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:56.176492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62188b94a70244f9975d34a2605a3d17', 'request_id': 'req-9529807b-1c01-4803-b903-31dcd5222268'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.191150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62188b94a70244f9975d34a2605a3d17', 'request_id': 'req-9529807b-1c01-4803-b903-31dcd5222268'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.210754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62188b94a70244f9975d34a2605a3d17', 'request_id': 'req-9529807b-1c01-4803-b903-31dcd5222268'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.231696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] The token cache is being invalidate because project 62188b94a70244f9975d34a2605a3d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:56.231979 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.232802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9529807b-1c01-4803-b903-31dcd5222268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.238484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 896/1806] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:55 2022] DELETE /identity/v3/projects/62188b94a70244f9975d34a2605a3d17 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:56.241162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d037b-2e35-4b5f-8f16-5a670db227ea None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:56.241338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d037b-2e35-4b5f-8f16-5a670db227ea None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:56.241749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d037b-2e35-4b5f-8f16-5a670db227ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:56.276188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:56.277125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:56.277414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:56.277695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] PATH_INFO: `/v3/projects/344f63da597c4f039723a694338d8885` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:56.278723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8a2d037b-2e35-4b5f-8f16-5a670db227ea None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:56.279391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=344f63da597c4f039723a694338d8885)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:56.281497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:56.282349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8a2d037b-2e35-4b5f-8f16-5a670db227ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:56.310625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 897/1807] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:04:56.317028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2f550-b059-4bf2-97db-2015e0bb0c28 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:56.318555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2f550-b059-4bf2-97db-2015e0bb0c28 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:56.318970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2f550-b059-4bf2-97db-2015e0bb0c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:56.332210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:56.364771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-63c2f550-b059-4bf2-97db-2015e0bb0c28 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:56.365994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-63c2f550-b059-4bf2-97db-2015e0bb0c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:56.387238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 898/1808] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:56.388739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2f550-b059-4bf2-97db-2015e0bb0c28 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:56.410353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344f63da597c4f039723a694338d8885', 'request_id': 'req-e048a4eb-e877-4b51-b713-c6a123ecaa4b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.418580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344f63da597c4f039723a694338d8885', 'request_id': 'req-e048a4eb-e877-4b51-b713-c6a123ecaa4b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.426384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344f63da597c4f039723a694338d8885', 'request_id': 'req-e048a4eb-e877-4b51-b713-c6a123ecaa4b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.435462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2f550-b059-4bf2-97db-2015e0bb0c28 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:56.445084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] The token cache is being invalidate because project 344f63da597c4f039723a694338d8885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:56.445262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.446038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e048a4eb-e877-4b51-b713-c6a123ecaa4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.453087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 911/1809] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:55 2022] DELETE /identity/v3/projects/344f63da597c4f039723a694338d8885 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:56.456085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc3947-d132-46a3-ba0c-cc17a0223951 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:56.456413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc3947-d132-46a3-ba0c-cc17a0223951 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:56.457162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc3947-d132-46a3-ba0c-cc17a0223951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:56.481809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-edbc3947-d132-46a3-ba0c-cc17a0223951 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:56.483067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-edbc3947-d132-46a3-ba0c-cc17a0223951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:56.613513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 912/1810] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:56.614567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc3947-d132-46a3-ba0c-cc17a0223951 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:56.621685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-edbc3947-d132-46a3-ba0c-cc17a0223951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:56.644455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc3947-d132-46a3-ba0c-cc17a0223951 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:56.658937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:56.659620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:56.659983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:56.660860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] PATH_INFO: `/v3/projects/25261eb9e0fc4f8c806b22d78981b934` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:56.662302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] RBAC: Authorizing `identity:delete_project(project_id=25261eb9e0fc4f8c806b22d78981b934)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:56.665392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:56.681955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:56.700711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81031e-c22d-4a5f-9860-3aff98cc9a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:56.701711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81031e-c22d-4a5f-9860-3aff98cc9a7a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:56.702151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81031e-c22d-4a5f-9860-3aff98cc9a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:56.702690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81031e-c22d-4a5f-9860-3aff98cc9a7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:56.703666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81031e-c22d-4a5f-9860-3aff98cc9a7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:56.711053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81031e-c22d-4a5f-9860-3aff98cc9a7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:56.713060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81031e-c22d-4a5f-9860-3aff98cc9a7a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:56.744138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 899/1811] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:56.749793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459524b-f555-4c70-b600-f9ce87ebf61f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:56.750126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459524b-f555-4c70-b600-f9ce87ebf61f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:56.750419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459524b-f555-4c70-b600-f9ce87ebf61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:56.759779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25261eb9e0fc4f8c806b22d78981b934', 'request_id': 'req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.779111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25261eb9e0fc4f8c806b22d78981b934', 'request_id': 'req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.789010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a459524b-f555-4c70-b600-f9ce87ebf61f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:56.789941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a459524b-f555-4c70-b600-f9ce87ebf61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:56.796120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25261eb9e0fc4f8c806b22d78981b934', 'request_id': 'req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.818519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] The token cache is being invalidate because project 25261eb9e0fc4f8c806b22d78981b934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:04:56.819199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.826202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:04:56.835471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 913/1812] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:04:56 2022] DELETE /identity/v3/projects/25261eb9e0fc4f8c806b22d78981b934 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:56.842279 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:56.848354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:56.949615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 900/1813] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:56.951268 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a277d9-ae1d-4c08-afdd-2bf023920f5a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:56.952218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a277d9-ae1d-4c08-afdd-2bf023920f5a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:56.952577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a277d9-ae1d-4c08-afdd-2bf023920f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:56.975757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8bef0-1e87-43fb-a32a-d42f3eaec19b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:56.980772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-04a277d9-ae1d-4c08-afdd-2bf023920f5a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:56.981483 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e7400-2c40-4b97-8c83-cf65e5f921d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:56.981993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-04a277d9-ae1d-4c08-afdd-2bf023920f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:56.982385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e7400-2c40-4b97-8c83-cf65e5f921d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:56.986528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e7400-2c40-4b97-8c83-cf65e5f921d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:56.986956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e7400-2c40-4b97-8c83-cf65e5f921d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:56.987860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e7400-2c40-4b97-8c83-cf65e5f921d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:56.991908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e7400-2c40-4b97-8c83-cf65e5f921d1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.007312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 901/1814] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:57.008453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a277d9-ae1d-4c08-afdd-2bf023920f5a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.039439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a277d9-ae1d-4c08-afdd-2bf023920f5a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.050321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 914/1815] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:56 2022] POST /identity/v3/projects => generated 377 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:57.050690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4621636-930f-4d4c-bc0b-e4dc6ad99b17 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.051000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4621636-930f-4d4c-bc0b-e4dc6ad99b17 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.051289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4621636-930f-4d4c-bc0b-e4dc6ad99b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.054816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759e7c9-994d-415b-af21-2f48cde6bd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.057353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759e7c9-994d-415b-af21-2f48cde6bd51 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.057627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759e7c9-994d-415b-af21-2f48cde6bd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.057907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759e7c9-994d-415b-af21-2f48cde6bd51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.062157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759e7c9-994d-415b-af21-2f48cde6bd51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:57.062506 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759e7c9-994d-415b-af21-2f48cde6bd51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.071254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759e7c9-994d-415b-af21-2f48cde6bd51 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.097905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e4621636-930f-4d4c-bc0b-e4dc6ad99b17 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:57.104828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e4621636-930f-4d4c-bc0b-e4dc6ad99b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:57.108812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 902/1816] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:57.109644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759e7c9-994d-415b-af21-2f48cde6bd51 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.132950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759e7c9-994d-415b-af21-2f48cde6bd51 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.243401 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ff5fb-55c7-4c0b-95ed-c006dd28f4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.243962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 915/1817] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:57.244337 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ff5fb-55c7-4c0b-95ed-c006dd28f4de admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.244727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ff5fb-55c7-4c0b-95ed-c006dd28f4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.244836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4621636-930f-4d4c-bc0b-e4dc6ad99b17 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.245184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ff5fb-55c7-4c0b-95ed-c006dd28f4de admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.245961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ff5fb-55c7-4c0b-95ed-c006dd28f4de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.248221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ff5fb-55c7-4c0b-95ed-c006dd28f4de admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.258808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4621636-930f-4d4c-bc0b-e4dc6ad99b17 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.266935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46679ad-4643-4449-a77c-b15bdfffa04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.267997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46679ad-4643-4449-a77c-b15bdfffa04c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.268546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46679ad-4643-4449-a77c-b15bdfffa04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.268698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46679ad-4643-4449-a77c-b15bdfffa04c admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.269470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46679ad-4643-4449-a77c-b15bdfffa04c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.274301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46679ad-4643-4449-a77c-b15bdfffa04c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.274509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:57.285059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 903/1818] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:57 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:57.289778 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ff5fb-55c7-4c0b-95ed-c006dd28f4de admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.304527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ff5fb-55c7-4c0b-95ed-c006dd28f4de admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.319421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 916/1819] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:57 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:57.319683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cef91c-982b-4b97-97d1-30cd4fdbf75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.320652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cef91c-982b-4b97-97d1-30cd4fdbf75a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.320652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cef91c-982b-4b97-97d1-30cd4fdbf75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.320904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46679ad-4643-4449-a77c-b15bdfffa04c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.321015 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cef91c-982b-4b97-97d1-30cd4fdbf75a admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.322075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cef91c-982b-4b97-97d1-30cd4fdbf75a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:04:57.322399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cef91c-982b-4b97-97d1-30cd4fdbf75a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.324144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cef91c-982b-4b97-97d1-30cd4fdbf75a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.326727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:57.341085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 904/1820] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:04:57.342145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cef91c-982b-4b97-97d1-30cd4fdbf75a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.355796 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cef91c-982b-4b97-97d1-30cd4fdbf75a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.412482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46679ad-4643-4449-a77c-b15bdfffa04c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.418643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70159adc-6995-47fd-b48e-f97d65743c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.419330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70159adc-6995-47fd-b48e-f97d65743c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.419622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70159adc-6995-47fd-b48e-f97d65743c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.419906 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70159adc-6995-47fd-b48e-f97d65743c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.420648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70159adc-6995-47fd-b48e-f97d65743c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.422300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70159adc-6995-47fd-b48e-f97d65743c2a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.435366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 917/1821] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:57.436423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70159adc-6995-47fd-b48e-f97d65743c2a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.461710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe635bc-bbd1-4959-8be0-c57d3db95ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.463302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70159adc-6995-47fd-b48e-f97d65743c2a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.467142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe635bc-bbd1-4959-8be0-c57d3db95ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.467432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe635bc-bbd1-4959-8be0-c57d3db95ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.467769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe635bc-bbd1-4959-8be0-c57d3db95ef5 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.468543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe635bc-bbd1-4959-8be0-c57d3db95ef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.471503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe635bc-bbd1-4959-8be0-c57d3db95ef5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.478959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d5ba5-da68-4e63-bde9-c358268fa12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.479849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5ba5-da68-4e63-bde9-c358268fa12f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.480184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5ba5-da68-4e63-bde9-c358268fa12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.480476 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5ba5-da68-4e63-bde9-c358268fa12f admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.481338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d5ba5-da68-4e63-bde9-c358268fa12f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.486715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d5ba5-da68-4e63-bde9-c358268fa12f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.511491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 905/1822] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:57 2022] POST /identity/v3/users => generated 294 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:57.512578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe635bc-bbd1-4959-8be0-c57d3db95ef5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.535579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe635bc-bbd1-4959-8be0-c57d3db95ef5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.543577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8ce28-24e6-42be-8b5f-e4c353e1985c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.544593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8ce28-24e6-42be-8b5f-e4c353e1985c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.544880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8ce28-24e6-42be-8b5f-e4c353e1985c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.545154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a8ce28-24e6-42be-8b5f-e4c353e1985c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.545899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8ce28-24e6-42be-8b5f-e4c353e1985c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.547670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a8ce28-24e6-42be-8b5f-e4c353e1985c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.555442 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 918/1823] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:57 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:57.556833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d5ba5-da68-4e63-bde9-c358268fa12f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.566725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 906/1824] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:57.568728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8ce28-24e6-42be-8b5f-e4c353e1985c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.588000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a8ce28-24e6-42be-8b5f-e4c353e1985c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.698996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d5ba5-da68-4e63-bde9-c358268fa12f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.699251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39fe8e-6f93-438e-bf5c-010bd106bc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.700316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39fe8e-6f93-438e-bf5c-010bd106bc7f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.701123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39fe8e-6f93-438e-bf5c-010bd106bc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.701597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39fe8e-6f93-438e-bf5c-010bd106bc7f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.702559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39fe8e-6f93-438e-bf5c-010bd106bc7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.705153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a18e1b-5160-48be-addf-4bc2acbfc73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.705915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a18e1b-5160-48be-addf-4bc2acbfc73d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.706564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a18e1b-5160-48be-addf-4bc2acbfc73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.706564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a18e1b-5160-48be-addf-4bc2acbfc73d admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.706925 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39fe8e-6f93-438e-bf5c-010bd106bc7f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.707466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a18e1b-5160-48be-addf-4bc2acbfc73d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.709561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a18e1b-5160-48be-addf-4bc2acbfc73d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.726938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 919/1825] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:57.726938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a18e1b-5160-48be-addf-4bc2acbfc73d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.746439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 907/1826] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:57 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:57.746439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39fe8e-6f93-438e-bf5c-010bd106bc7f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.772124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a18e1b-5160-48be-addf-4bc2acbfc73d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.780590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46950b9-974f-4875-88f3-86fd725698c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.781725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46950b9-974f-4875-88f3-86fd725698c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.782020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46950b9-974f-4875-88f3-86fd725698c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.782378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46950b9-974f-4875-88f3-86fd725698c9 admin admin] PATH_INFO: `/v3/projects/491c4328c56a4565abc99f963d7a785c/users/f86be24085e243ed98dbb195675a5d69/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.785980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39fe8e-6f93-438e-bf5c-010bd106bc7f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:57.827416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46950b9-974f-4875-88f3-86fd725698c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=491c4328c56a4565abc99f963d7a785c, user_id=f86be24085e243ed98dbb195675a5d69, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.830860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46950b9-974f-4875-88f3-86fd725698c9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.832309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bb1a3-695f-4ff7-9f61-87d4afd43df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:57.839494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bb1a3-695f-4ff7-9f61-87d4afd43df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.840505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bb1a3-695f-4ff7-9f61-87d4afd43df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.841147 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bb1a3-695f-4ff7-9f61-87d4afd43df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.844888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993bb1a3-695f-4ff7-9f61-87d4afd43df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:57.850650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993bb1a3-695f-4ff7-9f61-87d4afd43df0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:57.860046 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 920/1827] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:57 2022] PUT /identity/v3/projects/491c4328c56a4565abc99f963d7a785c/users/f86be24085e243ed98dbb195675a5d69/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:57.861205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46950b9-974f-4875-88f3-86fd725698c9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:57.894574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 908/1828] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:57.902667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f1f2b-7c2b-453a-9ead-81c7e00f1f8e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:57.902971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f1f2b-7c2b-453a-9ead-81c7e00f1f8e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:57.903262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f1f2b-7c2b-453a-9ead-81c7e00f1f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:57.970679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:57.980544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fa2f1f2b-7c2b-453a-9ead-81c7e00f1f8e None None] MFA Rules not processed for user `f86be24085e243ed98dbb195675a5d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:57.982545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fa2f1f2b-7c2b-453a-9ead-81c7e00f1f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:58.006208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46950b9-974f-4875-88f3-86fd725698c9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:58.019616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da711a59-a8d7-4d2f-a427-4f878486b758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:58.020510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da711a59-a8d7-4d2f-a427-4f878486b758 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:58.020804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da711a59-a8d7-4d2f-a427-4f878486b758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:58.021084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da711a59-a8d7-4d2f-a427-4f878486b758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:58.021822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da711a59-a8d7-4d2f-a427-4f878486b758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:58.023473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da711a59-a8d7-4d2f-a427-4f878486b758 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:58.031028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:58.069690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 921/1829] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:58.071260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da711a59-a8d7-4d2f-a427-4f878486b758 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:58.104880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da711a59-a8d7-4d2f-a427-4f878486b758 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:58.114187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 909/1830] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:58.115129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f1f2b-7c2b-453a-9ead-81c7e00f1f8e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:58.124782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0be90-3c39-43dc-bf00-92f31a7f7b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:58.128683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0be90-3c39-43dc-bf00-92f31a7f7b3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:58.128972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0be90-3c39-43dc-bf00-92f31a7f7b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:58.129256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0be90-3c39-43dc-bf00-92f31a7f7b3e admin admin] PATH_INFO: `/v3/projects/3e4d5a0c18654526bf1ca4eb2c224513/users/4516902bfab24328bf0906197ba89b11/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:58.133180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f1f2b-7c2b-453a-9ead-81c7e00f1f8e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:58.139536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e923f-1172-423d-a23b-97ccebd2ae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:58.140567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e923f-1172-423d-a23b-97ccebd2ae26 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:58.142872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e923f-1172-423d-a23b-97ccebd2ae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:58.143330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e923f-1172-423d-a23b-97ccebd2ae26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:58.144399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e923f-1172-423d-a23b-97ccebd2ae26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:58.146508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e923f-1172-423d-a23b-97ccebd2ae26 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:58.178968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 910/1831] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:58.180583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e923f-1172-423d-a23b-97ccebd2ae26 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:58.204317 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0be90-3c39-43dc-bf00-92f31a7f7b3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4d5a0c18654526bf1ca4eb2c224513, user_id=4516902bfab24328bf0906197ba89b11, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:58.205252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e923f-1172-423d-a23b-97ccebd2ae26 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:58.207111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb0be90-3c39-43dc-bf00-92f31a7f7b3e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:58.217165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b3bc1-4f43-4cac-b74b-f9d4bee5391c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:58.218124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3bc1-4f43-4cac-b74b-f9d4bee5391c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:58.220180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3bc1-4f43-4cac-b74b-f9d4bee5391c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:58.220642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3bc1-4f43-4cac-b74b-f9d4bee5391c admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:58.222125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b3bc1-4f43-4cac-b74b-f9d4bee5391c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:58.227452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b3bc1-4f43-4cac-b74b-f9d4bee5391c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:58.242595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 922/1832] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:58 2022] PUT /identity/v3/projects/3e4d5a0c18654526bf1ca4eb2c224513/users/4516902bfab24328bf0906197ba89b11/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:58.256639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0be90-3c39-43dc-bf00-92f31a7f7b3e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:58.275052 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 911/1833] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:58 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:58.276451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c908a9-6745-42c5-ad0e-523a12854d95 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:58.279433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c908a9-6745-42c5-ad0e-523a12854d95 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:58.280197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c908a9-6745-42c5-ad0e-523a12854d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:58.342842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-39c908a9-6745-42c5-ad0e-523a12854d95 None None] MFA Rules not processed for user `4516902bfab24328bf0906197ba89b11`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:58.351492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-39c908a9-6745-42c5-ad0e-523a12854d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:58.404019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb0be90-3c39-43dc-bf00-92f31a7f7b3e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:58.557255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 912/1834] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:58.557255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c908a9-6745-42c5-ad0e-523a12854d95 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:58.559695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75692772-d4bf-4eba-a704-e8fe1628d4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:58.581275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75692772-d4bf-4eba-a704-e8fe1628d4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:58.581656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75692772-d4bf-4eba-a704-e8fe1628d4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:58.582089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75692772-d4bf-4eba-a704-e8fe1628d4d4 admin admin] PATH_INFO: `/v3/projects/f18356dd4066451e848fe84dfd533592/users/7c4964f576e84f98a9263ed936966e74/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:58.587807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c908a9-6745-42c5-ad0e-523a12854d95 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:58.604972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d61586-4a47-441b-9227-5a5387dc1c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:58.605681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d61586-4a47-441b-9227-5a5387dc1c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:58.605970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d61586-4a47-441b-9227-5a5387dc1c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:58.606895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d61586-4a47-441b-9227-5a5387dc1c9a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:58.612216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d61586-4a47-441b-9227-5a5387dc1c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:58.614201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d61586-4a47-441b-9227-5a5387dc1c9a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:58.626345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75692772-d4bf-4eba-a704-e8fe1628d4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18356dd4066451e848fe84dfd533592, user_id=7c4964f576e84f98a9263ed936966e74, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:58.631961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75692772-d4bf-4eba-a704-e8fe1628d4d4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:58.686926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 923/1835] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:58 2022] PUT /identity/v3/projects/f18356dd4066451e848fe84dfd533592/users/7c4964f576e84f98a9263ed936966e74/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:58.686926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75692772-d4bf-4eba-a704-e8fe1628d4d4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:58.695109 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 913/1836] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:58 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:58.695109 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e5cd4-c0e3-4181-b6f2-02f3667ab4cc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:58.695109 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e5cd4-c0e3-4181-b6f2-02f3667ab4cc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:58.709109 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e5cd4-c0e3-4181-b6f2-02f3667ab4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:58.770046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b90e5cd4-c0e3-4181-b6f2-02f3667ab4cc None None] MFA Rules not processed for user `7c4964f576e84f98a9263ed936966e74`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:58.771639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b90e5cd4-c0e3-4181-b6f2-02f3667ab4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:58.809532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:58.823742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75692772-d4bf-4eba-a704-e8fe1628d4d4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:58.835410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1de722-90d8-4f0d-ab61-bb4aec0941c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:58.836091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1de722-90d8-4f0d-ab61-bb4aec0941c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:58.836379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1de722-90d8-4f0d-ab61-bb4aec0941c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:58.836688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1de722-90d8-4f0d-ab61-bb4aec0941c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:58.837726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1de722-90d8-4f0d-ab61-bb4aec0941c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:58.840334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1de722-90d8-4f0d-ab61-bb4aec0941c4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:58.936910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 924/1837] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:58 2022] POST /identity/v3/projects => generated 387 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:58.938073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1de722-90d8-4f0d-ab61-bb4aec0941c4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:58.963340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 914/1838] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:58.970683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e5cd4-c0e3-4181-b6f2-02f3667ab4cc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:58.977111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:58.989251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e5cd4-c0e3-4181-b6f2-02f3667ab4cc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.039593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1de722-90d8-4f0d-ab61-bb4aec0941c4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.060472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c23a5-c1e5-40fa-bbf0-a077ae485df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.061489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c23a5-c1e5-40fa-bbf0-a077ae485df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.061834 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c23a5-c1e5-40fa-bbf0-a077ae485df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.062125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c23a5-c1e5-40fa-bbf0-a077ae485df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.062963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c23a5-c1e5-40fa-bbf0-a077ae485df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.064762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c23a5-c1e5-40fa-bbf0-a077ae485df9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.094586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 925/1839] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:59.096185 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c23a5-c1e5-40fa-bbf0-a077ae485df9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.118857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c23a5-c1e5-40fa-bbf0-a077ae485df9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.133065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ddf6c-c2ca-4f7f-9309-c270cd31d459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.133065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ddf6c-c2ca-4f7f-9309-c270cd31d459 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.133065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ddf6c-c2ca-4f7f-9309-c270cd31d459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.133065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ddf6c-c2ca-4f7f-9309-c270cd31d459 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.133481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ddf6c-c2ca-4f7f-9309-c270cd31d459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.133550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa094a4-8641-4d1d-8a15-3d33cb17874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.133635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ddf6c-c2ca-4f7f-9309-c270cd31d459 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.145656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa094a4-8641-4d1d-8a15-3d33cb17874f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.150741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa094a4-8641-4d1d-8a15-3d33cb17874f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.151138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa094a4-8641-4d1d-8a15-3d33cb17874f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.153581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa094a4-8641-4d1d-8a15-3d33cb17874f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.155679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa094a4-8641-4d1d-8a15-3d33cb17874f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.184944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 926/1840] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:59 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:59.188111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ddf6c-c2ca-4f7f-9309-c270cd31d459 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.202512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 915/1841] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:58 2022] POST /identity/v3/users => generated 300 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:59.203776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa094a4-8641-4d1d-8a15-3d33cb17874f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.228298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa094a4-8641-4d1d-8a15-3d33cb17874f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.323318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ddf6c-c2ca-4f7f-9309-c270cd31d459 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.326921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72c1c2-01e3-4022-91bc-2c34318d5705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.327495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72c1c2-01e3-4022-91bc-2c34318d5705 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.327778 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72c1c2-01e3-4022-91bc-2c34318d5705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.328057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72c1c2-01e3-4022-91bc-2c34318d5705 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.328749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72c1c2-01e3-4022-91bc-2c34318d5705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.331607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ce652-eb5e-4835-93a5-159458619f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.332597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ce652-eb5e-4835-93a5-159458619f94 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.333163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ce652-eb5e-4835-93a5-159458619f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.333650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ce652-eb5e-4835-93a5-159458619f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.334624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ce652-eb5e-4835-93a5-159458619f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.336503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ce652-eb5e-4835-93a5-159458619f94 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.344585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72c1c2-01e3-4022-91bc-2c34318d5705 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.354905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 927/1842] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:59.356311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ce652-eb5e-4835-93a5-159458619f94 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.367900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ce652-eb5e-4835-93a5-159458619f94 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.375771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a9e1d-8d4b-4495-8c6b-923207278b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.377011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a9e1d-8d4b-4495-8c6b-923207278b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.377297 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a9e1d-8d4b-4495-8c6b-923207278b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.377575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a9e1d-8d4b-4495-8c6b-923207278b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.378351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a9e1d-8d4b-4495-8c6b-923207278b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.380247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 916/1843] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:59 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:59.380370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a9e1d-8d4b-4495-8c6b-923207278b1a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.382818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72c1c2-01e3-4022-91bc-2c34318d5705 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.393840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 928/1844] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:59.394607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a9e1d-8d4b-4495-8c6b-923207278b1a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.395926 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72c1c2-01e3-4022-91bc-2c34318d5705 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.401914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e96987-7676-4514-b875-2c826f5343eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.402929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e96987-7676-4514-b875-2c826f5343eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.403217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e96987-7676-4514-b875-2c826f5343eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.404499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e96987-7676-4514-b875-2c826f5343eb admin admin] PATH_INFO: `/v3/projects/c5388488bd054f7faa4c8c16adf19876/users/4936be9aa5dd44b2a64761aec1fe100c/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.406180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a9e1d-8d4b-4495-8c6b-923207278b1a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.412437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef53a7-7485-4256-9998-60c08b563348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.413121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef53a7-7485-4256-9998-60c08b563348 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.413403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef53a7-7485-4256-9998-60c08b563348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.413671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef53a7-7485-4256-9998-60c08b563348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.414884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef53a7-7485-4256-9998-60c08b563348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.416452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef53a7-7485-4256-9998-60c08b563348 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.422829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e96987-7676-4514-b875-2c826f5343eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5388488bd054f7faa4c8c16adf19876, user_id=4936be9aa5dd44b2a64761aec1fe100c, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.425654 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e96987-7676-4514-b875-2c826f5343eb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.429676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 929/1845] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:59.430630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef53a7-7485-4256-9998-60c08b563348 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.443266 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef53a7-7485-4256-9998-60c08b563348 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.450853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 917/1846] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:59 2022] PUT /identity/v3/projects/c5388488bd054f7faa4c8c16adf19876/users/4936be9aa5dd44b2a64761aec1fe100c/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:59.453434 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e96987-7676-4514-b875-2c826f5343eb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.481954 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:59.529630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:04:59.531031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b918e9-6626-4cca-9ba8-13728bd8437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.532014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b918e9-6626-4cca-9ba8-13728bd8437f admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.532338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b918e9-6626-4cca-9ba8-13728bd8437f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.532621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b918e9-6626-4cca-9ba8-13728bd8437f admin admin] PATH_INFO: `/v3/projects/96cd530db5864a5b8aa51eeaea477061/users/128e20d6e7204f9b953b761ea34722c9/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.534345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e96987-7676-4514-b875-2c826f5343eb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.544120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ac7c9-4f4e-46b4-bac2-d3cbbb0bbaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.545106 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ac7c9-4f4e-46b4-bac2-d3cbbb0bbaa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.545461 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ac7c9-4f4e-46b4-bac2-d3cbbb0bbaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.545807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ac7c9-4f4e-46b4-bac2-d3cbbb0bbaa0 admin admin] PATH_INFO: `/v3/projects/be94d4ab502a4bf5957f2f6115030877/users/12be77cb1fdc40b0bdfa82274f9daf9d/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.550314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b918e9-6626-4cca-9ba8-13728bd8437f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96cd530db5864a5b8aa51eeaea477061, user_id=128e20d6e7204f9b953b761ea34722c9, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.553263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b918e9-6626-4cca-9ba8-13728bd8437f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.567688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ac7c9-4f4e-46b4-bac2-d3cbbb0bbaa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be94d4ab502a4bf5957f2f6115030877, user_id=12be77cb1fdc40b0bdfa82274f9daf9d, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.571377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ac7c9-4f4e-46b4-bac2-d3cbbb0bbaa0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.573417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 930/1847] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:59 2022] PUT /identity/v3/projects/96cd530db5864a5b8aa51eeaea477061/users/128e20d6e7204f9b953b761ea34722c9/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:59.574910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dcfdf-a2af-486b-b4c0-d5309267a150 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.575213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dcfdf-a2af-486b-b4c0-d5309267a150 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.575582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31dcfdf-a2af-486b-b4c0-d5309267a150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.592153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 918/1848] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:59 2022] PUT /identity/v3/projects/be94d4ab502a4bf5957f2f6115030877/users/12be77cb1fdc40b0bdfa82274f9daf9d/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:59.595276 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ac7c9-4f4e-46b4-bac2-d3cbbb0bbaa0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.616535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e31dcfdf-a2af-486b-b4c0-d5309267a150 None None] MFA Rules not processed for user `4936be9aa5dd44b2a64761aec1fe100c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:04:59.617833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e31dcfdf-a2af-486b-b4c0-d5309267a150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:04:59.690156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ac7c9-4f4e-46b4-bac2-d3cbbb0bbaa0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.695206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4275df1-2221-4205-b9d3-4f02a21fb0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.695885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4275df1-2221-4205-b9d3-4f02a21fb0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.696309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4275df1-2221-4205-b9d3-4f02a21fb0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.696486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4275df1-2221-4205-b9d3-4f02a21fb0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.697247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4275df1-2221-4205-b9d3-4f02a21fb0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.698932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4275df1-2221-4205-b9d3-4f02a21fb0d2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.705928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 931/1849] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:04:59.707208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dcfdf-a2af-486b-b4c0-d5309267a150 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.714663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 919/1850] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:59.715563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4275df1-2221-4205-b9d3-4f02a21fb0d2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.724693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31dcfdf-a2af-486b-b4c0-d5309267a150 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.725698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4275df1-2221-4205-b9d3-4f02a21fb0d2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.730686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760122c6-c74b-4e57-a782-6b3793dfcb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.731353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760122c6-c74b-4e57-a782-6b3793dfcb77 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.731640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760122c6-c74b-4e57-a782-6b3793dfcb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.731927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760122c6-c74b-4e57-a782-6b3793dfcb77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.732283 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f69b-d4d7-465d-8b60-ddd337c96b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.732644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760122c6-c74b-4e57-a782-6b3793dfcb77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.733490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f69b-d4d7-465d-8b60-ddd337c96b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.734166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f69b-d4d7-465d-8b60-ddd337c96b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.734608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760122c6-c74b-4e57-a782-6b3793dfcb77 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.734843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f69b-d4d7-465d-8b60-ddd337c96b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.736540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29f69b-d4d7-465d-8b60-ddd337c96b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.738322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29f69b-d4d7-465d-8b60-ddd337c96b43 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.758472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 932/1851] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:04:59.762728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f69b-d4d7-465d-8b60-ddd337c96b43 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.775820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f69b-d4d7-465d-8b60-ddd337c96b43 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:04:59.775992 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 920/1852] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:04:59 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:04:59.781200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760122c6-c74b-4e57-a782-6b3793dfcb77 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.891238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b4987-931a-4931-856f-30a72bbf0621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:04:59.892288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b4987-931a-4931-856f-30a72bbf0621 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:04:59.892680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b4987-931a-4931-856f-30a72bbf0621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:04:59.893036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b4987-931a-4931-856f-30a72bbf0621 admin admin] PATH_INFO: `/v3/projects/96cd530db5864a5b8aa51eeaea477061/users/128e20d6e7204f9b953b761ea34722c9/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:04:59.896235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b4987-931a-4931-856f-30a72bbf0621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96cd530db5864a5b8aa51eeaea477061, user_id=128e20d6e7204f9b953b761ea34722c9, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:04:59.898578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b4987-931a-4931-856f-30a72bbf0621 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:04:59.945748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 933/1853] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:59 2022] PUT /identity/v3/projects/96cd530db5864a5b8aa51eeaea477061/users/128e20d6e7204f9b953b761ea34722c9/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:04:59.945748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b4987-931a-4931-856f-30a72bbf0621 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:04:59.946025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760122c6-c74b-4e57-a782-6b3793dfcb77 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:00.030473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:00.064763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8fa862-c12e-458a-9c5c-8f1252340d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:00.066220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fa862-c12e-458a-9c5c-8f1252340d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.066641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fa862-c12e-458a-9c5c-8f1252340d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.067023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8fa862-c12e-458a-9c5c-8f1252340d9f admin admin] PATH_INFO: `/v3/projects/be94d4ab502a4bf5957f2f6115030877/users/12be77cb1fdc40b0bdfa82274f9daf9d/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.070875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fa862-c12e-458a-9c5c-8f1252340d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=be94d4ab502a4bf5957f2f6115030877, user_id=12be77cb1fdc40b0bdfa82274f9daf9d, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:00.074437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8fa862-c12e-458a-9c5c-8f1252340d9f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:00.078327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:00.093627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b4987-931a-4931-856f-30a72bbf0621 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:00.100474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 921/1854] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:04:59 2022] PUT /identity/v3/projects/be94d4ab502a4bf5957f2f6115030877/users/12be77cb1fdc40b0bdfa82274f9daf9d/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:00.107003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9975f4a-d9f7-4297-880e-72502e07f936 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.107485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9975f4a-d9f7-4297-880e-72502e07f936 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.109056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9975f4a-d9f7-4297-880e-72502e07f936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.146096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e9975f4a-d9f7-4297-880e-72502e07f936 None None] MFA Rules not processed for user `128e20d6e7204f9b953b761ea34722c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:00.147798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e9975f4a-d9f7-4297-880e-72502e07f936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:00.192490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb47815-cc0f-4279-baa3-f0206b9a1bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:00.193165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47815-cc0f-4279-baa3-f0206b9a1bbe admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.193449 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47815-cc0f-4279-baa3-f0206b9a1bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.193726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47815-cc0f-4279-baa3-f0206b9a1bbe admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.194560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb47815-cc0f-4279-baa3-f0206b9a1bbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:00.196566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb47815-cc0f-4279-baa3-f0206b9a1bbe admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:00.216763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 934/1855] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:04:59 2022] POST /identity/v3/users => generated 294 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:00.218123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60098b0a-7dd0-4581-8836-70b8e557d7f9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.218414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60098b0a-7dd0-4581-8836-70b8e557d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.218713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60098b0a-7dd0-4581-8836-70b8e557d7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.236419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 922/1856] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:00.237308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9975f4a-d9f7-4297-880e-72502e07f936 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:00.246996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9975f4a-d9f7-4297-880e-72502e07f936 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:00.251803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-60098b0a-7dd0-4581-8836-70b8e557d7f9 None None] MFA Rules not processed for user `12be77cb1fdc40b0bdfa82274f9daf9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:00.252445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7088cc-bdc1-4875-85e4-db974de49f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:00.253023 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-60098b0a-7dd0-4581-8836-70b8e557d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:00.253138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7088cc-bdc1-4875-85e4-db974de49f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.253446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7088cc-bdc1-4875-85e4-db974de49f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.253812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7088cc-bdc1-4875-85e4-db974de49f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.254574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7088cc-bdc1-4875-85e4-db974de49f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:00.256236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7088cc-bdc1-4875-85e4-db974de49f7f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:00.270897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 923/1857] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:00.270897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7088cc-bdc1-4875-85e4-db974de49f7f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:00.273187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8f7088cc-bdc1-4875-85e4-db974de49f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:00.337942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 935/1858] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:00.338908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60098b0a-7dd0-4581-8836-70b8e557d7f9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:00.351152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60098b0a-7dd0-4581-8836-70b8e557d7f9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:00.356009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6c258-ca52-4eaf-a334-70cb58f4fac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:00.356908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c258-ca52-4eaf-a334-70cb58f4fac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.357195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c258-ca52-4eaf-a334-70cb58f4fac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.357474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6c258-ca52-4eaf-a334-70cb58f4fac8 admin admin] PATH_INFO: `/v3/projects/f65c76fdd33845fab0d6540ec33edf6f/users/f9c1aee0f94a414d9604fe3832962102/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.362709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7088cc-bdc1-4875-85e4-db974de49f7f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:00.367760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07840df-8e77-425a-a4c0-f80b6624bf7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:00.368604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07840df-8e77-425a-a4c0-f80b6624bf7d service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.369048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07840df-8e77-425a-a4c0-f80b6624bf7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.369394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07840df-8e77-425a-a4c0-f80b6624bf7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.372469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6c258-ca52-4eaf-a334-70cb58f4fac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f65c76fdd33845fab0d6540ec33edf6f, user_id=f9c1aee0f94a414d9604fe3832962102, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:00.375475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6c258-ca52-4eaf-a334-70cb58f4fac8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:00.381380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07840df-8e77-425a-a4c0-f80b6624bf7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:00.383347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07840df-8e77-425a-a4c0-f80b6624bf7d service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:00.389271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 936/1859] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:00 2022] PUT /identity/v3/projects/f65c76fdd33845fab0d6540ec33edf6f/users/f9c1aee0f94a414d9604fe3832962102/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:00.389979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6c258-ca52-4eaf-a334-70cb58f4fac8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:00.392187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-91e6c258-ca52-4eaf-a334-70cb58f4fac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:00.397253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 924/1860] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:00.398931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07840df-8e77-425a-a4c0-f80b6624bf7d service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:00.412265 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07840df-8e77-425a-a4c0-f80b6624bf7d service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:00.426148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:00.476714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6c258-ca52-4eaf-a334-70cb58f4fac8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:00.480647 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3954e-9087-48be-961e-d2ff10ff7fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:00.481304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3954e-9087-48be-961e-d2ff10ff7fba service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.481593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3954e-9087-48be-961e-d2ff10ff7fba service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.482056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3954e-9087-48be-961e-d2ff10ff7fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.497120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cca909-de39-475a-b4e6-5372fc9fd402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:00.498051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cca909-de39-475a-b4e6-5372fc9fd402 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.498935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cca909-de39-475a-b4e6-5372fc9fd402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.499457 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cca909-de39-475a-b4e6-5372fc9fd402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.500445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cca909-de39-475a-b4e6-5372fc9fd402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:00.502515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cca909-de39-475a-b4e6-5372fc9fd402 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:00.515042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 925/1861] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:00.521360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cca909-de39-475a-b4e6-5372fc9fd402 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:00.538374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cca909-de39-475a-b4e6-5372fc9fd402 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:00.546062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9715943-ced1-4938-a811-6c25c5fe2a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:00.547011 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9715943-ced1-4938-a811-6c25c5fe2a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.547331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9715943-ced1-4938-a811-6c25c5fe2a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.547573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9715943-ced1-4938-a811-6c25c5fe2a6d admin admin] PATH_INFO: `/v3/projects/f65c76fdd33845fab0d6540ec33edf6f/users/f9c1aee0f94a414d9604fe3832962102/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.552335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9715943-ced1-4938-a811-6c25c5fe2a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f65c76fdd33845fab0d6540ec33edf6f, user_id=f9c1aee0f94a414d9604fe3832962102, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:00.561149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9715943-ced1-4938-a811-6c25c5fe2a6d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:00.563970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:00.573765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3954e-9087-48be-961e-d2ff10ff7fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:00.575848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3954e-9087-48be-961e-d2ff10ff7fba service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:00.576993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 926/1862] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:00 2022] PUT /identity/v3/projects/f65c76fdd33845fab0d6540ec33edf6f/users/f9c1aee0f94a414d9604fe3832962102/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:00.578011 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9715943-ced1-4938-a811-6c25c5fe2a6d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:00.590881 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 937/1863] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:00.592664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ae56c-089e-46e3-bbc9-8c56a7a2c971 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.592965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ae56c-089e-46e3-bbc9-8c56a7a2c971 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.593247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ae56c-089e-46e3-bbc9-8c56a7a2c971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.636510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2b1ae56c-089e-46e3-bbc9-8c56a7a2c971 None None] MFA Rules not processed for user `f9c1aee0f94a414d9604fe3832962102`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:00.637916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2b1ae56c-089e-46e3-bbc9-8c56a7a2c971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:00.674708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9715943-ced1-4938-a811-6c25c5fe2a6d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:00.690738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:00.693310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.693453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.693695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] PATH_INFO: `/v3/users/4516902bfab24328bf0906197ba89b11` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.695380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4516902bfab24328bf0906197ba89b11)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:00.699870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:00.781749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 938/1864] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:00.794501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ae56c-089e-46e3-bbc9-8c56a7a2c971 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:00.798390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4516902bfab24328bf0906197ba89b11'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:00.804384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4516902bfab24328bf0906197ba89b11'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:00.822540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4516902bfab24328bf0906197ba89b11'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:00.825459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4516902bfab24328bf0906197ba89b11'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:00.842543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-532ac27c-61cd-4c89-b000-272f641fc55f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4516902bfab24328bf0906197ba89b11'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:00.860496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 927/1865] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:00 2022] DELETE /identity/v3/users/4516902bfab24328bf0906197ba89b11 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:00.864645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23097186-7e77-4e08-abb2-3a640f675893 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.865216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23097186-7e77-4e08-abb2-3a640f675893 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.865789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23097186-7e77-4e08-abb2-3a640f675893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.875702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ae56c-089e-46e3-bbc9-8c56a7a2c971 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:00.886958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:00.898167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-23097186-7e77-4e08-abb2-3a640f675893 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:00.899423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-23097186-7e77-4e08-abb2-3a640f675893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:00.966871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9db690-cbe4-4e15-8b66-b43d0f4d9048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:00.967583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9db690-cbe4-4e15-8b66-b43d0f4d9048 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:00.967935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9db690-cbe4-4e15-8b66-b43d0f4d9048 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:00.968299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9db690-cbe4-4e15-8b66-b43d0f4d9048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:00.970470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9a9db690-cbe4-4e15-8b66-b43d0f4d9048 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:00.986284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 928/1866] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:00.993687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23097186-7e77-4e08-abb2-3a640f675893 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:00.995857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-23097186-7e77-4e08-abb2-3a640f675893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:01.026047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:01.065417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9db690-cbe4-4e15-8b66-b43d0f4d9048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:01.067797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9db690-cbe4-4e15-8b66-b43d0f4d9048 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:01.081454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 939/1867] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:05:01.090712 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9db690-cbe4-4e15-8b66-b43d0f4d9048 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:01.090835 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23097186-7e77-4e08-abb2-3a640f675893 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:01.092544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9a9db690-cbe4-4e15-8b66-b43d0f4d9048 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:01.094687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02456765-4a96-4f16-8cb2-37c12c3974ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:01.095428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02456765-4a96-4f16-8cb2-37c12c3974ce service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:01.095719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02456765-4a96-4f16-8cb2-37c12c3974ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:01.096003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02456765-4a96-4f16-8cb2-37c12c3974ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:01.113292 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9db690-cbe4-4e15-8b66-b43d0f4d9048 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:01.119674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:01.120248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02456765-4a96-4f16-8cb2-37c12c3974ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:01.120760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:01.121148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:01.121542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] PATH_INFO: `/v3/users/7c4964f576e84f98a9263ed936966e74` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:01.122573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02456765-4a96-4f16-8cb2-37c12c3974ce service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:01.123184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c4964f576e84f98a9263ed936966e74)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:01.125998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:01.136387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 929/1868] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:01.203558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c4964f576e84f98a9263ed936966e74'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.226791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c4964f576e84f98a9263ed936966e74'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.243227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c4964f576e84f98a9263ed936966e74'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.244023 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c4964f576e84f98a9263ed936966e74'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.255420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c4964f576e84f98a9263ed936966e74'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.263296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 940/1869] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:01 2022] DELETE /identity/v3/users/7c4964f576e84f98a9263ed936966e74 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:01.268081 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec8750-c6de-4e58-ae08-dc9cc1cc5ece None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:01.268400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec8750-c6de-4e58-ae08-dc9cc1cc5ece None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:01.268683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec8750-c6de-4e58-ae08-dc9cc1cc5ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:01.290670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-96ec8750-c6de-4e58-ae08-dc9cc1cc5ece None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:01.292139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-96ec8750-c6de-4e58-ae08-dc9cc1cc5ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:01.374763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 930/1870] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:01.385323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:01.386936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:01.446939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:01.455864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec8750-c6de-4e58-ae08-dc9cc1cc5ece None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:01.457349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-96ec8750-c6de-4e58-ae08-dc9cc1cc5ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:01.471976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d611f7-11d1-4bcc-835c-ea92f8f930b2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:01.478846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec8750-c6de-4e58-ae08-dc9cc1cc5ece None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:01.488255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459bcc4-b66d-4308-8cca-41de51af4c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:01.489071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459bcc4-b66d-4308-8cca-41de51af4c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:01.489234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459bcc4-b66d-4308-8cca-41de51af4c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:01.489482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459bcc4-b66d-4308-8cca-41de51af4c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:01.494304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:01.495875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:01.496160 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:01.496437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] PATH_INFO: `/v3/projects/3e4d5a0c18654526bf1ca4eb2c224513` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:01.497650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4d5a0c18654526bf1ca4eb2c224513)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:01.499969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:01.514252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459bcc4-b66d-4308-8cca-41de51af4c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:01.516970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459bcc4-b66d-4308-8cca-41de51af4c15 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:01.538707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 941/1871] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:01.574637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4d5a0c18654526bf1ca4eb2c224513', 'request_id': 'req-d26aacbd-96a0-4176-a649-75704544ecd4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.580447 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4d5a0c18654526bf1ca4eb2c224513', 'request_id': 'req-d26aacbd-96a0-4176-a649-75704544ecd4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.587564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4d5a0c18654526bf1ca4eb2c224513', 'request_id': 'req-d26aacbd-96a0-4176-a649-75704544ecd4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.601636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] The token cache is being invalidate because project 3e4d5a0c18654526bf1ca4eb2c224513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:01.601999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.602765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.604625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 931/1872] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:01 2022] DELETE /identity/v3/projects/3e4d5a0c18654526bf1ca4eb2c224513 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:01.608358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459bcc4-b66d-4308-8cca-41de51af4c15 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:01.621595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459bcc4-b66d-4308-8cca-41de51af4c15 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:01.693867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:01.694952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:01.695761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:01.696651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] PATH_INFO: `/v3/users/128e20d6e7204f9b953b761ea34722c9` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:01.697331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] RBAC: Authorizing `identity:delete_user(user_id=128e20d6e7204f9b953b761ea34722c9)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:01.699573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:01.796764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128e20d6e7204f9b953b761ea34722c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.824205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128e20d6e7204f9b953b761ea34722c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.840385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128e20d6e7204f9b953b761ea34722c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.841410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128e20d6e7204f9b953b761ea34722c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.854742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128e20d6e7204f9b953b761ea34722c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:01.864764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 942/1873] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:01 2022] DELETE /identity/v3/users/128e20d6e7204f9b953b761ea34722c9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:02.349308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:02.350870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:02.352421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:02.354217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:02.375120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26aacbd-96a0-4176-a649-75704544ecd4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:02.425385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:02.457862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:02.458758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:02.459096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:02.459366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] PATH_INFO: `/v3/projects/f18356dd4066451e848fe84dfd533592` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:02.460584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18356dd4066451e848fe84dfd533592)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:02.463415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:02.473265 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:02.480241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3e5c0-1d4f-491f-a3a2-de46ef58a6db admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:02.487789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:02.489653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:02.490009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:02.490351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] PATH_INFO: `/v3/projects/96cd530db5864a5b8aa51eeaea477061` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:02.491922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96cd530db5864a5b8aa51eeaea477061)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:02.495008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:02.536500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18356dd4066451e848fe84dfd533592', 'request_id': 'req-6b557793-678b-4f9e-96fa-a65421667024'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.543040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18356dd4066451e848fe84dfd533592', 'request_id': 'req-6b557793-678b-4f9e-96fa-a65421667024'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.554574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18356dd4066451e848fe84dfd533592', 'request_id': 'req-6b557793-678b-4f9e-96fa-a65421667024'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.566094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96cd530db5864a5b8aa51eeaea477061', 'request_id': 'req-12373731-662a-434e-9ef5-523650961ba0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.568565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] The token cache is being invalidate because project f18356dd4066451e848fe84dfd533592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:02.568854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.569544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.571202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 932/1874] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:02 2022] DELETE /identity/v3/projects/f18356dd4066451e848fe84dfd533592 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:02.575116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:02.576211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96cd530db5864a5b8aa51eeaea477061', 'request_id': 'req-12373731-662a-434e-9ef5-523650961ba0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.576642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:02.582835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96cd530db5864a5b8aa51eeaea477061', 'request_id': 'req-12373731-662a-434e-9ef5-523650961ba0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.599584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] The token cache is being invalidate because project 96cd530db5864a5b8aa51eeaea477061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:02.599995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.600692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12373731-662a-434e-9ef5-523650961ba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.605471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 943/1875] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:02 2022] DELETE /identity/v3/projects/96cd530db5864a5b8aa51eeaea477061 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:02.607063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c8c66-e65a-4a88-abf2-c90c7d2222bc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:02.607548 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c8c66-e65a-4a88-abf2-c90c7d2222bc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:02.607743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3c8c66-e65a-4a88-abf2-c90c7d2222bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:02.628863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4e3c8c66-e65a-4a88-abf2-c90c7d2222bc None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:02.630220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4e3c8c66-e65a-4a88-abf2-c90c7d2222bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:02.656329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b557793-678b-4f9e-96fa-a65421667024 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:02.709359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 944/1876] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:02.716868 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a4fea-8b9c-4145-a854-933fcb23a47c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:02.717168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a4fea-8b9c-4145-a854-933fcb23a47c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:02.717456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a4fea-8b9c-4145-a854-933fcb23a47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:02.718726 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:02.720191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:02.720543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:02.720754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] PATH_INFO: `/v3/users/12be77cb1fdc40b0bdfa82274f9daf9d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:02.721955 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12be77cb1fdc40b0bdfa82274f9daf9d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:02.723970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:02.743389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-224a4fea-8b9c-4145-a854-933fcb23a47c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:02.744635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-224a4fea-8b9c-4145-a854-933fcb23a47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:02.753464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 945/1877] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:02.754230 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a4fea-8b9c-4145-a854-933fcb23a47c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:02.755235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:02.755601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-224a4fea-8b9c-4145-a854-933fcb23a47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:02.790087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12be77cb1fdc40b0bdfa82274f9daf9d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.791649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12be77cb1fdc40b0bdfa82274f9daf9d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.803572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12be77cb1fdc40b0bdfa82274f9daf9d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.804305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12be77cb1fdc40b0bdfa82274f9daf9d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.837275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12be77cb1fdc40b0bdfa82274f9daf9d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:02.848089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a4fea-8b9c-4145-a854-933fcb23a47c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:02.852728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64917963-7a21-4b49-b906-7bb0b404c1a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:02.853398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64917963-7a21-4b49-b906-7bb0b404c1a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:02.853681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64917963-7a21-4b49-b906-7bb0b404c1a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:02.853960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64917963-7a21-4b49-b906-7bb0b404c1a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:02.857273 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-64917963-7a21-4b49-b906-7bb0b404c1a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:02.869630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 933/1878] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:02 2022] DELETE /identity/v3/users/12be77cb1fdc40b0bdfa82274f9daf9d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:02.869630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:02.869630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:02.937816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:02.974200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54599c-5032-4204-b74f-058e2870fdb7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:02.979678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64917963-7a21-4b49-b906-7bb0b404c1a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:02.983212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64917963-7a21-4b49-b906-7bb0b404c1a3 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:02.988185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ac3a-f51e-4ab4-878a-fa733dcedec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:02.988185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ac3a-f51e-4ab4-878a-fa733dcedec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:02.989811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ac3a-f51e-4ab4-878a-fa733dcedec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:02.990169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ac3a-f51e-4ab4-878a-fa733dcedec6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:02.992175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45ac3a-f51e-4ab4-878a-fa733dcedec6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:02.992855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45ac3a-f51e-4ab4-878a-fa733dcedec6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:02.999848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45ac3a-f51e-4ab4-878a-fa733dcedec6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.004454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 946/1879] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:03.043986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 934/1880] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:05:03.049850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64917963-7a21-4b49-b906-7bb0b404c1a3 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.067287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64917963-7a21-4b49-b906-7bb0b404c1a3 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:03.076194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864a346-0831-4989-81de-2a111dee9c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:03.076799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864a346-0831-4989-81de-2a111dee9c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.077542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864a346-0831-4989-81de-2a111dee9c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.077542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864a346-0831-4989-81de-2a111dee9c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.078078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864a346-0831-4989-81de-2a111dee9c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:03.081578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864a346-0831-4989-81de-2a111dee9c75 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.132882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 947/1881] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:03 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:03.133794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ac3a-f51e-4ab4-878a-fa733dcedec6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.149296 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ac3a-f51e-4ab4-878a-fa733dcedec6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:03.274587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568d7dc-a9ce-4bae-82d6-f1bcc9e3d238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:03.276365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568d7dc-a9ce-4bae-82d6-f1bcc9e3d238 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.276917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568d7dc-a9ce-4bae-82d6-f1bcc9e3d238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.277221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568d7dc-a9ce-4bae-82d6-f1bcc9e3d238 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.279308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568d7dc-a9ce-4bae-82d6-f1bcc9e3d238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:03.285586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568d7dc-a9ce-4bae-82d6-f1bcc9e3d238 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.340306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 935/1882] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:03 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:05:03.343987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864a346-0831-4989-81de-2a111dee9c75 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.358212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864a346-0831-4989-81de-2a111dee9c75 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:03.376844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dfeaa-45d4-4468-98e3-9a4993d7eabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:03.378042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dfeaa-45d4-4468-98e3-9a4993d7eabe admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.378584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dfeaa-45d4-4468-98e3-9a4993d7eabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.379573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15dfeaa-45d4-4468-98e3-9a4993d7eabe admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.380696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15dfeaa-45d4-4468-98e3-9a4993d7eabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:03.385897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15dfeaa-45d4-4468-98e3-9a4993d7eabe admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.406540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 948/1883] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:05:03.410023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568d7dc-a9ce-4bae-82d6-f1bcc9e3d238 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.422100 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dfeaa-45d4-4468-98e3-9a4993d7eabe admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.424665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d15dfeaa-45d4-4468-98e3-9a4993d7eabe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:03.443658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568d7dc-a9ce-4bae-82d6-f1bcc9e3d238 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:03.449119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b741c-d3d4-4967-99ed-5d7a7634e8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:03.458835 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b741c-d3d4-4967-99ed-5d7a7634e8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.458961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15dfeaa-45d4-4468-98e3-9a4993d7eabe admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:03.459805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b741c-d3d4-4967-99ed-5d7a7634e8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.459805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b741c-d3d4-4967-99ed-5d7a7634e8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.460180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b741c-d3d4-4967-99ed-5d7a7634e8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:03.461829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b741c-d3d4-4967-99ed-5d7a7634e8d2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.471451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:03.472482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.472767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.473190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] PATH_INFO: `/v3/projects/be94d4ab502a4bf5957f2f6115030877` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.476840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 936/1884] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:05:03.476973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be94d4ab502a4bf5957f2f6115030877)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:03.478194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b741c-d3d4-4967-99ed-5d7a7634e8d2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.480933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.490663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b741c-d3d4-4967-99ed-5d7a7634e8d2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:03.496601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fac0dd-e13b-419c-a48b-15c96f0fd8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:03.498242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fac0dd-e13b-419c-a48b-15c96f0fd8d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.498843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fac0dd-e13b-419c-a48b-15c96f0fd8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.499132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fac0dd-e13b-419c-a48b-15c96f0fd8d3 admin admin] PATH_INFO: `/v3/projects/4c461358e18842479381971cb910c921/users/a17883821ba545afbda701e990d47b2c/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.513016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:03.525637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fac0dd-e13b-419c-a48b-15c96f0fd8d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c461358e18842479381971cb910c921, user_id=a17883821ba545afbda701e990d47b2c, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:03.529804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fac0dd-e13b-419c-a48b-15c96f0fd8d3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.555362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 937/1885] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:03 2022] PUT /identity/v3/projects/4c461358e18842479381971cb910c921/users/a17883821ba545afbda701e990d47b2c/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:03.556351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be94d4ab502a4bf5957f2f6115030877', 'request_id': 'req-63231e69-5672-4aba-948b-5d5a2f67dd96'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:03.565417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be94d4ab502a4bf5957f2f6115030877', 'request_id': 'req-63231e69-5672-4aba-948b-5d5a2f67dd96'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:03.565574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e76c8-2645-4925-96ad-04962a50d06e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.567753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e76c8-2645-4925-96ad-04962a50d06e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.568116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e76c8-2645-4925-96ad-04962a50d06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.571989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be94d4ab502a4bf5957f2f6115030877', 'request_id': 'req-63231e69-5672-4aba-948b-5d5a2f67dd96'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:03.601296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] The token cache is being invalidate because project be94d4ab502a4bf5957f2f6115030877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:03.601591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:03.602293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-63231e69-5672-4aba-948b-5d5a2f67dd96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:03.605376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 949/1886] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:03 2022] DELETE /identity/v3/projects/be94d4ab502a4bf5957f2f6115030877 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:03.610765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e144dd4-c465-4d84-83ce-b2741679484f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.610915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f83e76c8-2645-4925-96ad-04962a50d06e None None] MFA Rules not processed for user `a17883821ba545afbda701e990d47b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:03.611264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e144dd4-c465-4d84-83ce-b2741679484f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.612012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e144dd4-c465-4d84-83ce-b2741679484f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.612177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f83e76c8-2645-4925-96ad-04962a50d06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:03.659452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9e144dd4-c465-4d84-83ce-b2741679484f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:03.660549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e144dd4-c465-4d84-83ce-b2741679484f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:03.706840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 938/1887] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:03.711324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e76c8-2645-4925-96ad-04962a50d06e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.712866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f83e76c8-2645-4925-96ad-04962a50d06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:03.749315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 950/1888] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:03.757827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419ae26-b810-4830-bb9b-c0e23133631d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.758271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419ae26-b810-4830-bb9b-c0e23133631d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.758609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419ae26-b810-4830-bb9b-c0e23133631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.781109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5419ae26-b810-4830-bb9b-c0e23133631d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:03.782381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5419ae26-b810-4830-bb9b-c0e23133631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:03.793760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 951/1889] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:03.794711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419ae26-b810-4830-bb9b-c0e23133631d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.806989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419ae26-b810-4830-bb9b-c0e23133631d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:03.810125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e76c8-2645-4925-96ad-04962a50d06e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:03.814250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998af94-c822-4ad5-b2f7-0d712583867d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:03.816677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59015d8-01a9-42c4-9604-a861cc221bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:03.817309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59015d8-01a9-42c4-9604-a861cc221bbd admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.817679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59015d8-01a9-42c4-9604-a861cc221bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.818040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59015d8-01a9-42c4-9604-a861cc221bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.818869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59015d8-01a9-42c4-9604-a861cc221bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:03.820507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998af94-c822-4ad5-b2f7-0d712583867d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.821047 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998af94-c822-4ad5-b2f7-0d712583867d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.821245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59015d8-01a9-42c4-9604-a861cc221bbd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.821921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998af94-c822-4ad5-b2f7-0d712583867d admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.823103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998af94-c822-4ad5-b2f7-0d712583867d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:03.823656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998af94-c822-4ad5-b2f7-0d712583867d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:03.825708 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998af94-c822-4ad5-b2f7-0d712583867d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.846978 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 952/1890] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:05:03.848008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998af94-c822-4ad5-b2f7-0d712583867d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.855363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 939/1891] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:03 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:05:03.858910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59015d8-01a9-42c4-9604-a861cc221bbd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.859974 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998af94-c822-4ad5-b2f7-0d712583867d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:03.864083 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:03.929897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59015d8-01a9-42c4-9604-a861cc221bbd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:03.930178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca656f-56bd-40a5-9c49-61dd03ed601f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:03.930915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca656f-56bd-40a5-9c49-61dd03ed601f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.931206 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca656f-56bd-40a5-9c49-61dd03ed601f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.931518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca656f-56bd-40a5-9c49-61dd03ed601f admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.932238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca656f-56bd-40a5-9c49-61dd03ed601f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:03.934148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca656f-56bd-40a5-9c49-61dd03ed601f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.934926 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3502b5e-98bb-4b5b-8787-d3ad5c7e88e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:03.935653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3502b5e-98bb-4b5b-8787-d3ad5c7e88e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:03.936000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3502b5e-98bb-4b5b-8787-d3ad5c7e88e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:03.936345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3502b5e-98bb-4b5b-8787-d3ad5c7e88e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:03.937091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3502b5e-98bb-4b5b-8787-d3ad5c7e88e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:03.939626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3502b5e-98bb-4b5b-8787-d3ad5c7e88e9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:03.947267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:03.964430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 940/1892] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:03 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:05:03.964430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3502b5e-98bb-4b5b-8787-d3ad5c7e88e9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:03.964676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 953/1893] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:03 2022] POST /identity/v3/projects => generated 369 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:05:03.964766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d3502b5e-98bb-4b5b-8787-d3ad5c7e88e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:03.966648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca656f-56bd-40a5-9c49-61dd03ed601f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.025029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3502b5e-98bb-4b5b-8787-d3ad5c7e88e9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.028673 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca656f-56bd-40a5-9c49-61dd03ed601f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.033407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb482ae-6a2d-47ca-b295-448818b2d813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.034073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb482ae-6a2d-47ca-b295-448818b2d813 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.034354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb482ae-6a2d-47ca-b295-448818b2d813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.034700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb482ae-6a2d-47ca-b295-448818b2d813 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.035382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb482ae-6a2d-47ca-b295-448818b2d813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.037301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb482ae-6a2d-47ca-b295-448818b2d813 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.057346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 954/1894] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:03 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:05:04.058079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb482ae-6a2d-47ca-b295-448818b2d813 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.068886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb482ae-6a2d-47ca-b295-448818b2d813 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.074691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf55e75-cfc0-4b92-80f1-f9f917684231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.075433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf55e75-cfc0-4b92-80f1-f9f917684231 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.075796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf55e75-cfc0-4b92-80f1-f9f917684231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.076146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf55e75-cfc0-4b92-80f1-f9f917684231 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.076953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf55e75-cfc0-4b92-80f1-f9f917684231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.078581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf55e75-cfc0-4b92-80f1-f9f917684231 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.078742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260feca-0751-4bba-8b41-078ce431467a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.080002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260feca-0751-4bba-8b41-078ce431467a service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.080439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260feca-0751-4bba-8b41-078ce431467a service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.080734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260feca-0751-4bba-8b41-078ce431467a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.082608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5260feca-0751-4bba-8b41-078ce431467a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:04.089292 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 955/1895] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:04.090084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf55e75-cfc0-4b92-80f1-f9f917684231 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.099618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf55e75-cfc0-4b92-80f1-f9f917684231 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.104168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a576d5-df8c-4fd8-84b9-36a972d9eae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.104794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a576d5-df8c-4fd8-84b9-36a972d9eae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.105077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a576d5-df8c-4fd8-84b9-36a972d9eae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.105355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a576d5-df8c-4fd8-84b9-36a972d9eae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.106056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a576d5-df8c-4fd8-84b9-36a972d9eae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.107649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a576d5-df8c-4fd8-84b9-36a972d9eae6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.121101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 956/1896] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:04.121101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a576d5-df8c-4fd8-84b9-36a972d9eae6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.131322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a576d5-df8c-4fd8-84b9-36a972d9eae6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.135910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef062654-4b80-46fc-80d6-28364a1ff78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.136529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef062654-4b80-46fc-80d6-28364a1ff78f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.136814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef062654-4b80-46fc-80d6-28364a1ff78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.137104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef062654-4b80-46fc-80d6-28364a1ff78f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.137824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef062654-4b80-46fc-80d6-28364a1ff78f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.139406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef062654-4b80-46fc-80d6-28364a1ff78f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.151159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 957/1897] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:04.152118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef062654-4b80-46fc-80d6-28364a1ff78f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.157317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260feca-0751-4bba-8b41-078ce431467a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.158915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260feca-0751-4bba-8b41-078ce431467a service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.162031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef062654-4b80-46fc-80d6-28364a1ff78f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.166957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d0bef-4f04-4425-b2e9-7cfbd07f7fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.167656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0bef-4f04-4425-b2e9-7cfbd07f7fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.167969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0bef-4f04-4425-b2e9-7cfbd07f7fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.168242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0bef-4f04-4425-b2e9-7cfbd07f7fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.168946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d0bef-4f04-4425-b2e9-7cfbd07f7fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.169053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 941/1898] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:04.169830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260feca-0751-4bba-8b41-078ce431467a service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.170587 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d0bef-4f04-4425-b2e9-7cfbd07f7fb8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.182364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 958/1899] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:04.187122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260feca-0751-4bba-8b41-078ce431467a service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.187452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d0bef-4f04-4425-b2e9-7cfbd07f7fb8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.192848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8442097-1f60-4024-8426-5cca36d55949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.193748 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8442097-1f60-4024-8426-5cca36d55949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.194034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8442097-1f60-4024-8426-5cca36d55949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.202766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8442097-1f60-4024-8426-5cca36d55949 admin admin] PATH_INFO: `/v3/projects/fcfa20f761ab4915affd8f0233a09e83/users/940dd9784c7749b1960050ee8b09daac/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.217974 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d0bef-4f04-4425-b2e9-7cfbd07f7fb8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.231228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8442097-1f60-4024-8426-5cca36d55949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcfa20f761ab4915affd8f0233a09e83, user_id=940dd9784c7749b1960050ee8b09daac, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.233365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e322f2e-d645-40e6-914c-b7b0bb9bd57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.234811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8442097-1f60-4024-8426-5cca36d55949 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.236162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322f2e-d645-40e6-914c-b7b0bb9bd57d admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.237408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322f2e-d645-40e6-914c-b7b0bb9bd57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.238230 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322f2e-d645-40e6-914c-b7b0bb9bd57d admin admin] PATH_INFO: `/v3/projects/0e9e3591b9684c73bff5a44b022a0bb2/users/204254a6025e42f098f2e41cfe435671/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.257559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 942/1900] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:04 2022] PUT /identity/v3/projects/fcfa20f761ab4915affd8f0233a09e83/users/940dd9784c7749b1960050ee8b09daac/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:04.269056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d287b3b-3697-49ae-b244-c6f9079406da None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.269586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d287b3b-3697-49ae-b244-c6f9079406da None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.270429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d287b3b-3697-49ae-b244-c6f9079406da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.296793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e322f2e-d645-40e6-914c-b7b0bb9bd57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9e3591b9684c73bff5a44b022a0bb2, user_id=204254a6025e42f098f2e41cfe435671, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.300841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e322f2e-d645-40e6-914c-b7b0bb9bd57d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.318509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6d287b3b-3697-49ae-b244-c6f9079406da None None] MFA Rules not processed for user `940dd9784c7749b1960050ee8b09daac`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:04.320557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6d287b3b-3697-49ae-b244-c6f9079406da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:04.332671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 959/1901] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:04 2022] PUT /identity/v3/projects/0e9e3591b9684c73bff5a44b022a0bb2/users/204254a6025e42f098f2e41cfe435671/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:04.344578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b52b37-28e4-404c-8e4f-81ebdce8b713 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.344883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b52b37-28e4-404c-8e4f-81ebdce8b713 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.345172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b52b37-28e4-404c-8e4f-81ebdce8b713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.368681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:04.389566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:04.404483 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b7b52b37-28e4-404c-8e4f-81ebdce8b713 None None] MFA Rules not processed for user `204254a6025e42f098f2e41cfe435671`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:04.405244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b7b52b37-28e4-404c-8e4f-81ebdce8b713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:04.483632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 943/1902] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:04.485655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d287b3b-3697-49ae-b244-c6f9079406da None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.581134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 960/1903] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:04.585653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b52b37-28e4-404c-8e4f-81ebdce8b713 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.612888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d287b3b-3697-49ae-b244-c6f9079406da None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.707761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b52b37-28e4-404c-8e4f-81ebdce8b713 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.713551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2189a-f49e-4b69-840d-ded282d665be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.714314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2189a-f49e-4b69-840d-ded282d665be admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.714851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2189a-f49e-4b69-840d-ded282d665be admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.715127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2189a-f49e-4b69-840d-ded282d665be admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.716052 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2189a-f49e-4b69-840d-ded282d665be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.718311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2189a-f49e-4b69-840d-ded282d665be admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.739568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ef0b4-004d-4b0a-80fd-23d980df24f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.740324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ef0b4-004d-4b0a-80fd-23d980df24f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.740690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ef0b4-004d-4b0a-80fd-23d980df24f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.741369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ef0b4-004d-4b0a-80fd-23d980df24f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.742391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ef0b4-004d-4b0a-80fd-23d980df24f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.745703 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ef0b4-004d-4b0a-80fd-23d980df24f7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.752989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 961/1904] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:04 2022] POST /identity/v3/projects => generated 367 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:04.757364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2189a-f49e-4b69-840d-ded282d665be admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.819869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 944/1905] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:04 2022] POST /identity/v3/projects => generated 387 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:05:04.824108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ef0b4-004d-4b0a-80fd-23d980df24f7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:04.863212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2189a-f49e-4b69-840d-ded282d665be admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.919591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:04.930227 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:04.939125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ef0b4-004d-4b0a-80fd-23d980df24f7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:04.947276 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e0a66-48f2-49f0-9efb-afbbbe49a61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.948701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e0a66-48f2-49f0-9efb-afbbbe49a61e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.949076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e0a66-48f2-49f0-9efb-afbbbe49a61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.949544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e0a66-48f2-49f0-9efb-afbbbe49a61e admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.950329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e0a66-48f2-49f0-9efb-afbbbe49a61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.953018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e0a66-48f2-49f0-9efb-afbbbe49a61e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:04.986712 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071aaec-573c-4674-b751-4147d69076de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:04.987534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071aaec-573c-4674-b751-4147d69076de admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:04.987967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071aaec-573c-4674-b751-4147d69076de admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:04.988350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071aaec-573c-4674-b751-4147d69076de admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:04.989639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071aaec-573c-4674-b751-4147d69076de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:04.993087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071aaec-573c-4674-b751-4147d69076de admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:05.003153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 945/1906] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:04 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:05:05.004935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e0a66-48f2-49f0-9efb-afbbbe49a61e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:05.028988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e0a66-48f2-49f0-9efb-afbbbe49a61e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:05.031268 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 962/1907] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:04 2022] POST /identity/v3/users => generated 290 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:05.037338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071aaec-573c-4674-b751-4147d69076de admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:05.041594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260ffc1-f1b5-48b0-b8d8-81cd261638f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:05.042421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260ffc1-f1b5-48b0-b8d8-81cd261638f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.042890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260ffc1-f1b5-48b0-b8d8-81cd261638f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.043295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260ffc1-f1b5-48b0-b8d8-81cd261638f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:05.044195 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260ffc1-f1b5-48b0-b8d8-81cd261638f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:05.045957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260ffc1-f1b5-48b0-b8d8-81cd261638f7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:05.064985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 946/1908] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:05.065933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071aaec-573c-4674-b751-4147d69076de admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:05.070868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260ffc1-f1b5-48b0-b8d8-81cd261638f7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:05.084528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9877f9-47b4-4002-abec-041acee95070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:05.086162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9877f9-47b4-4002-abec-041acee95070 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.087845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9877f9-47b4-4002-abec-041acee95070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.088313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9877f9-47b4-4002-abec-041acee95070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:05.089224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9877f9-47b4-4002-abec-041acee95070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:05.090719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260ffc1-f1b5-48b0-b8d8-81cd261638f7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:05.093641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9877f9-47b4-4002-abec-041acee95070 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:05.099382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c2995-be8f-47df-b63b-693518351a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:05.100399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c2995-be8f-47df-b63b-693518351a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.100929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c2995-be8f-47df-b63b-693518351a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.101361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c2995-be8f-47df-b63b-693518351a6c admin admin] PATH_INFO: `/v3/projects/5ea1a14ad7244d01aef20f5e3f0f49ec/users/a5ba731c1d3a43c797816be3e0d9b4f6/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:05.114978 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 963/1909] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:05.119943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9877f9-47b4-4002-abec-041acee95070 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:05.136515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c2995-be8f-47df-b63b-693518351a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ea1a14ad7244d01aef20f5e3f0f49ec, user_id=a5ba731c1d3a43c797816be3e0d9b4f6, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:05.137882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9877f9-47b4-4002-abec-041acee95070 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:05.141631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c2995-be8f-47df-b63b-693518351a6c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:05.143531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b615ec-c8d1-48dd-8096-c39a73ad0115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:05.144718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b615ec-c8d1-48dd-8096-c39a73ad0115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.145002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b615ec-c8d1-48dd-8096-c39a73ad0115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.145792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b615ec-c8d1-48dd-8096-c39a73ad0115 admin admin] PATH_INFO: `/v3/projects/3c9e1685d2bd4ccd9b707adf006ee624/users/dc662cd4c61f4dd7bc8526d9218d5a87/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:05.164702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 947/1910] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:05 2022] PUT /identity/v3/projects/5ea1a14ad7244d01aef20f5e3f0f49ec/users/a5ba731c1d3a43c797816be3e0d9b4f6/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:05.169191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c2995-be8f-47df-b63b-693518351a6c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:05.175674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b615ec-c8d1-48dd-8096-c39a73ad0115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9e1685d2bd4ccd9b707adf006ee624, user_id=dc662cd4c61f4dd7bc8526d9218d5a87, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:05.179451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b615ec-c8d1-48dd-8096-c39a73ad0115 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:05.204302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 964/1911] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:05 2022] PUT /identity/v3/projects/3c9e1685d2bd4ccd9b707adf006ee624/users/dc662cd4c61f4dd7bc8526d9218d5a87/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:05.208566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b615ec-c8d1-48dd-8096-c39a73ad0115 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:05.253219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c2995-be8f-47df-b63b-693518351a6c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:05.300845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b615ec-c8d1-48dd-8096-c39a73ad0115 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:05.313586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f3522-3341-462f-a8fc-af7ac1f2c79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:05.314303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f3522-3341-462f-a8fc-af7ac1f2c79c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.314617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f3522-3341-462f-a8fc-af7ac1f2c79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.314897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f3522-3341-462f-a8fc-af7ac1f2c79c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:05.315623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f3522-3341-462f-a8fc-af7ac1f2c79c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:05.323937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f3522-3341-462f-a8fc-af7ac1f2c79c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:05.344427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 965/1912] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:05.346172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f3522-3341-462f-a8fc-af7ac1f2c79c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:05.349014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aeda1-9797-4a60-9de3-c12756c3e47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:05.351937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aeda1-9797-4a60-9de3-c12756c3e47b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.354775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aeda1-9797-4a60-9de3-c12756c3e47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.355238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0aeda1-9797-4a60-9de3-c12756c3e47b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:05.356664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aeda1-9797-4a60-9de3-c12756c3e47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:05.363927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f3522-3341-462f-a8fc-af7ac1f2c79c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:05.366267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0aeda1-9797-4a60-9de3-c12756c3e47b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:05.374065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd661778-fc8f-403d-ac21-293c5c5fd85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:05.375436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd661778-fc8f-403d-ac21-293c5c5fd85b admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.375776 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd661778-fc8f-403d-ac21-293c5c5fd85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.376092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd661778-fc8f-403d-ac21-293c5c5fd85b admin admin] PATH_INFO: `/v3/projects/3c9e1685d2bd4ccd9b707adf006ee624/users/dc662cd4c61f4dd7bc8526d9218d5a87/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:05.378743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd661778-fc8f-403d-ac21-293c5c5fd85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9e1685d2bd4ccd9b707adf006ee624, user_id=dc662cd4c61f4dd7bc8526d9218d5a87, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:05.381788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd661778-fc8f-403d-ac21-293c5c5fd85b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:05.390148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 948/1913] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:05.392365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aeda1-9797-4a60-9de3-c12756c3e47b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:05.408166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 966/1914] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:05 2022] PUT /identity/v3/projects/3c9e1685d2bd4ccd9b707adf006ee624/users/dc662cd4c61f4dd7bc8526d9218d5a87/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:05.419699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12631e15-e040-4aad-ac96-b7c70b56b49d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.420636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12631e15-e040-4aad-ac96-b7c70b56b49d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.421154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12631e15-e040-4aad-ac96-b7c70b56b49d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:05.430419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0aeda1-9797-4a60-9de3-c12756c3e47b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:05.511953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-12631e15-e040-4aad-ac96-b7c70b56b49d None None] MFA Rules not processed for user `dc662cd4c61f4dd7bc8526d9218d5a87`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:05.517711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-12631e15-e040-4aad-ac96-b7c70b56b49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:05.544417 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:05.566238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3941e1-84a1-42f2-be87-589ff10c76fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:05.567189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3941e1-84a1-42f2-be87-589ff10c76fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.567468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3941e1-84a1-42f2-be87-589ff10c76fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.567740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3941e1-84a1-42f2-be87-589ff10c76fa admin admin] PATH_INFO: `/v3/projects/5ea1a14ad7244d01aef20f5e3f0f49ec/users/a5ba731c1d3a43c797816be3e0d9b4f6/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:05.575294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3941e1-84a1-42f2-be87-589ff10c76fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ea1a14ad7244d01aef20f5e3f0f49ec, user_id=a5ba731c1d3a43c797816be3e0d9b4f6, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:05.581425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3941e1-84a1-42f2-be87-589ff10c76fa admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:05.626651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 949/1915] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:05 2022] PUT /identity/v3/projects/5ea1a14ad7244d01aef20f5e3f0f49ec/users/a5ba731c1d3a43c797816be3e0d9b4f6/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:05.628695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ec8a2-8876-4fe8-9579-d1cc9ffdf117 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.629089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ec8a2-8876-4fe8-9579-d1cc9ffdf117 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.629472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ec8a2-8876-4fe8-9579-d1cc9ffdf117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:05.677082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:05.692630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-515ec8a2-8876-4fe8-9579-d1cc9ffdf117 None None] MFA Rules not processed for user `a5ba731c1d3a43c797816be3e0d9b4f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:05.697624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-515ec8a2-8876-4fe8-9579-d1cc9ffdf117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:05.700315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 967/1916] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:05.714039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12631e15-e040-4aad-ac96-b7c70b56b49d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:05.715605 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-12631e15-e040-4aad-ac96-b7c70b56b49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:05.874004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12631e15-e040-4aad-ac96-b7c70b56b49d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:05.910428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 950/1917] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:05.940580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ec8a2-8876-4fe8-9579-d1cc9ffdf117 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:05.970702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e44f1-65d3-49ca-9798-32f229fd2ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:05.971380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e44f1-65d3-49ca-9798-32f229fd2ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:05.971670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e44f1-65d3-49ca-9798-32f229fd2ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:05.972113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e44f1-65d3-49ca-9798-32f229fd2ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:06.015720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ec8a2-8876-4fe8-9579-d1cc9ffdf117 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:06.020097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b61b3-1e2e-4b73-8141-8234e7a099bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:06.020747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b61b3-1e2e-4b73-8141-8234e7a099bc service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:06.021035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b61b3-1e2e-4b73-8141-8234e7a099bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:06.021316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b61b3-1e2e-4b73-8141-8234e7a099bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:06.032419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b61b3-1e2e-4b73-8141-8234e7a099bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:06.034121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b61b3-1e2e-4b73-8141-8234e7a099bc service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:06.046121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 951/1918] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:05:06.049620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e44f1-65d3-49ca-9798-32f229fd2ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:06.051114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e44f1-65d3-49ca-9798-32f229fd2ef7 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:06.070621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 968/1919] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:05:06.746982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b61b3-1e2e-4b73-8141-8234e7a099bc service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:06.757150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b61b3-1e2e-4b73-8141-8234e7a099bc service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:06.763424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95d648-3934-4b43-8673-aa99557a4a0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:06.764157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95d648-3934-4b43-8673-aa99557a4a0d service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:06.764568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95d648-3934-4b43-8673-aa99557a4a0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:06.764910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95d648-3934-4b43-8673-aa99557a4a0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:06.828533 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95d648-3934-4b43-8673-aa99557a4a0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:06.830031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95d648-3934-4b43-8673-aa99557a4a0d service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:06.839212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 952/1920] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:07.626729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e44f1-65d3-49ca-9798-32f229fd2ef7 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:07.688767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e44f1-65d3-49ca-9798-32f229fd2ef7 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:07.693222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:07.694085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:07.694366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:07.694677 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] PATH_INFO: `/v3/users/204254a6025e42f098f2e41cfe435671` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:07.695850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=204254a6025e42f098f2e41cfe435671)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:07.697765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:07.702414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:07.747793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204254a6025e42f098f2e41cfe435671'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:07.760768 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204254a6025e42f098f2e41cfe435671'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:07.768575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204254a6025e42f098f2e41cfe435671'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:07.769231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204254a6025e42f098f2e41cfe435671'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:07.776418 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204254a6025e42f098f2e41cfe435671'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:07.782320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 969/1921] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:07 2022] DELETE /identity/v3/users/204254a6025e42f098f2e41cfe435671 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:07.786803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e20b5-d0cf-4f3b-8959-a11191858672 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:07.787287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e20b5-d0cf-4f3b-8959-a11191858672 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:07.787392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e20b5-d0cf-4f3b-8959-a11191858672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:07.804207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-480e20b5-d0cf-4f3b-8959-a11191858672 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:07.805410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-480e20b5-d0cf-4f3b-8959-a11191858672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:07.814197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:07.865331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 953/1922] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:07.875076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:07.877115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:07.935918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b624da-79d2-464b-bc06-b393f71808a9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:07.940243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef7152-a03c-4d81-af2e-c7eb4f2c3142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:07.940888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef7152-a03c-4d81-af2e-c7eb4f2c3142 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:07.941174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef7152-a03c-4d81-af2e-c7eb4f2c3142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:07.941453 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef7152-a03c-4d81-af2e-c7eb4f2c3142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:07.950374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef7152-a03c-4d81-af2e-c7eb4f2c3142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:07.951831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef7152-a03c-4d81-af2e-c7eb4f2c3142 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:07.961219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 970/1923] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:08.237126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e20b5-d0cf-4f3b-8959-a11191858672 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:08.238729 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-480e20b5-d0cf-4f3b-8959-a11191858672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:08.252548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e20b5-d0cf-4f3b-8959-a11191858672 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:08.278947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:08.278947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:08.278947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:08.278947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] PATH_INFO: `/v3/projects/0e9e3591b9684c73bff5a44b022a0bb2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:08.279933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e9e3591b9684c73bff5a44b022a0bb2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:08.283714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:08.367099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e9e3591b9684c73bff5a44b022a0bb2', 'request_id': 'req-b523bdea-7b0e-41b1-856d-85867532e61d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:08.373096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e9e3591b9684c73bff5a44b022a0bb2', 'request_id': 'req-b523bdea-7b0e-41b1-856d-85867532e61d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:08.380850 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e9e3591b9684c73bff5a44b022a0bb2', 'request_id': 'req-b523bdea-7b0e-41b1-856d-85867532e61d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:08.395466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] The token cache is being invalidate because project 0e9e3591b9684c73bff5a44b022a0bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:08.395765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:08.396439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:08.398099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 954/1924] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:08 2022] DELETE /identity/v3/projects/0e9e3591b9684c73bff5a44b022a0bb2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:08.402164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef7152-a03c-4d81-af2e-c7eb4f2c3142 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:08.413411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef7152-a03c-4d81-af2e-c7eb4f2c3142 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:08.483949 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:08.484952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:08.485926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:08.486289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] PATH_INFO: `/v3/users/dc662cd4c61f4dd7bc8526d9218d5a87` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:08.487702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc662cd4c61f4dd7bc8526d9218d5a87)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:08.489725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:08.519566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:08.540834 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc662cd4c61f4dd7bc8526d9218d5a87'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:08.553774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc662cd4c61f4dd7bc8526d9218d5a87'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:08.563903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc662cd4c61f4dd7bc8526d9218d5a87'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:08.569057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc662cd4c61f4dd7bc8526d9218d5a87'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:08.580420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-eb117c1e-eff7-4470-b477-917b75302cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc662cd4c61f4dd7bc8526d9218d5a87'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:08.586861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 971/1925] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:08 2022] DELETE /identity/v3/users/dc662cd4c61f4dd7bc8526d9218d5a87 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:08.976110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:08.978298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:08.991531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523bdea-7b0e-41b1-856d-85867532e61d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:09.054131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:09.055033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.055319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.055598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] PATH_INFO: `/v3/projects/3c9e1685d2bd4ccd9b707adf006ee624` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.056785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9e1685d2bd4ccd9b707adf006ee624)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:09.058762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:09.102678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:09.110407 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9e1685d2bd4ccd9b707adf006ee624', 'request_id': 'req-df094138-6061-4173-86ce-2a25ee0276b2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:09.115773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9e1685d2bd4ccd9b707adf006ee624', 'request_id': 'req-df094138-6061-4173-86ce-2a25ee0276b2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:09.121687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9e1685d2bd4ccd9b707adf006ee624', 'request_id': 'req-df094138-6061-4173-86ce-2a25ee0276b2'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:09.133846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] The token cache is being invalidate because project 3c9e1685d2bd4ccd9b707adf006ee624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:09.134130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:09.134754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-df094138-6061-4173-86ce-2a25ee0276b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:09.136287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 955/1926] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:08 2022] DELETE /identity/v3/projects/3c9e1685d2bd4ccd9b707adf006ee624 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:09.141301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d63de-d2ac-44f0-be13-d7425120727f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.141767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d63de-d2ac-44f0-be13-d7425120727f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.141871 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d63de-d2ac-44f0-be13-d7425120727f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.159716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-710d63de-d2ac-44f0-be13-d7425120727f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:09.160903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-710d63de-d2ac-44f0-be13-d7425120727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:09.218947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 972/1927] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:09.226770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6012d62-91a5-4077-91ca-b136e035eb35 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.227064 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6012d62-91a5-4077-91ca-b136e035eb35 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.227349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6012d62-91a5-4077-91ca-b136e035eb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.245459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a6012d62-91a5-4077-91ca-b136e035eb35 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:09.246674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a6012d62-91a5-4077-91ca-b136e035eb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:09.255606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 956/1928] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:09.259847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d63de-d2ac-44f0-be13-d7425120727f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:09.270344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d63de-d2ac-44f0-be13-d7425120727f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:09.277467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0c8e5-3640-42ca-9977-369c2be977d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:09.278100 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0c8e5-3640-42ca-9977-369c2be977d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.278391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0c8e5-3640-42ca-9977-369c2be977d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.278720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0c8e5-3640-42ca-9977-369c2be977d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.279455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0c8e5-3640-42ca-9977-369c2be977d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:09.279766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0c8e5-3640-42ca-9977-369c2be977d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:09.281353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0c8e5-3640-42ca-9977-369c2be977d8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:09.296855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 973/1929] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:05:09.298061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6012d62-91a5-4077-91ca-b136e035eb35 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:09.308073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6012d62-91a5-4077-91ca-b136e035eb35 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:09.318407 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543d1ba-30b3-4b0a-9c52-f51fdc603456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:09.319243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543d1ba-30b3-4b0a-9c52-f51fdc603456 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.319534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543d1ba-30b3-4b0a-9c52-f51fdc603456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.319946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543d1ba-30b3-4b0a-9c52-f51fdc603456 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.320650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543d1ba-30b3-4b0a-9c52-f51fdc603456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:09.325091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543d1ba-30b3-4b0a-9c52-f51fdc603456 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:09.377107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 957/1930] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:09 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:09.378763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0c8e5-3640-42ca-9977-369c2be977d8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:09.391412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0c8e5-3640-42ca-9977-369c2be977d8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:09.476990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bc6f1-7eb2-425d-8da4-20b3626512f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:09.477791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bc6f1-7eb2-425d-8da4-20b3626512f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.478176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bc6f1-7eb2-425d-8da4-20b3626512f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.478580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bc6f1-7eb2-425d-8da4-20b3626512f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.479424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bc6f1-7eb2-425d-8da4-20b3626512f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:09.486884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85bc6f1-7eb2-425d-8da4-20b3626512f9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:09.519158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 974/1931] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:09 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:09.524520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543d1ba-30b3-4b0a-9c52-f51fdc603456 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:09.536244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543d1ba-30b3-4b0a-9c52-f51fdc603456 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:09.541137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012f5fb-07a1-4954-be42-c497321d78f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:09.541815 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012f5fb-07a1-4954-be42-c497321d78f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.542104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012f5fb-07a1-4954-be42-c497321d78f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.542387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012f5fb-07a1-4954-be42-c497321d78f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.543251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012f5fb-07a1-4954-be42-c497321d78f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:09.544902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012f5fb-07a1-4954-be42-c497321d78f3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:09.560723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 958/1932] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:09.562719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bc6f1-7eb2-425d-8da4-20b3626512f9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:09.575803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85bc6f1-7eb2-425d-8da4-20b3626512f9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:09.582981 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d9a6b-f91f-4ccb-98e6-2205b22e31d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:09.583639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d9a6b-f91f-4ccb-98e6-2205b22e31d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.583927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d9a6b-f91f-4ccb-98e6-2205b22e31d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.584209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d9a6b-f91f-4ccb-98e6-2205b22e31d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.584939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d9a6b-f91f-4ccb-98e6-2205b22e31d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:09.586628 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d9a6b-f91f-4ccb-98e6-2205b22e31d7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:09.597802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 975/1933] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:05:09.601926 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012f5fb-07a1-4954-be42-c497321d78f3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:09.618763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012f5fb-07a1-4954-be42-c497321d78f3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:09.624299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc4a96-5f1b-4881-b735-2f0a6d202be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:09.626182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc4a96-5f1b-4881-b735-2f0a6d202be5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.626595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc4a96-5f1b-4881-b735-2f0a6d202be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.626944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc4a96-5f1b-4881-b735-2f0a6d202be5 admin admin] PATH_INFO: `/v3/projects/0f2612edd696457597e8719fc18383ba/users/6e10b02ca643416fa138141d32b96091/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.652846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc4a96-5f1b-4881-b735-2f0a6d202be5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2612edd696457597e8719fc18383ba, user_id=6e10b02ca643416fa138141d32b96091, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:09.655961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc4a96-5f1b-4881-b735-2f0a6d202be5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:09.672896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 959/1934] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:09 2022] PUT /identity/v3/projects/0f2612edd696457597e8719fc18383ba/users/6e10b02ca643416fa138141d32b96091/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:09.684314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d646ef4-6cd6-4e10-92f4-6ddfbf366776 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.684662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d646ef4-6cd6-4e10-92f4-6ddfbf366776 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.684950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d646ef4-6cd6-4e10-92f4-6ddfbf366776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.706020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:09.717841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1d646ef4-6cd6-4e10-92f4-6ddfbf366776 None None] MFA Rules not processed for user `6e10b02ca643416fa138141d32b96091`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:09.719196 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1d646ef4-6cd6-4e10-92f4-6ddfbf366776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:09.791813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 976/1935] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:09.796084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc4a96-5f1b-4881-b735-2f0a6d202be5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:09.880978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc4a96-5f1b-4881-b735-2f0a6d202be5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:09.885718 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8a17-a092-4301-b5af-6106f3af1b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:09.886409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8a17-a092-4301-b5af-6106f3af1b72 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.886733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8a17-a092-4301-b5af-6106f3af1b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.887010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c8a17-a092-4301-b5af-6106f3af1b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.887697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c8a17-a092-4301-b5af-6106f3af1b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:09.889651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c8a17-a092-4301-b5af-6106f3af1b72 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:09.915458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 960/1936] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:09 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:09.919600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d646ef4-6cd6-4e10-92f4-6ddfbf366776 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:09.930947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d646ef4-6cd6-4e10-92f4-6ddfbf366776 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:09.987459 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ec0b3-9777-4a85-913e-ae378490b3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:09.988044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ec0b3-9777-4a85-913e-ae378490b3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:09.988346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ec0b3-9777-4a85-913e-ae378490b3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:09.988608 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802ec0b3-9777-4a85-913e-ae378490b3e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:09.989291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ec0b3-9777-4a85-913e-ae378490b3e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:09.991251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802ec0b3-9777-4a85-913e-ae378490b3e8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:10.011994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 977/1937] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:09 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:10.014053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8a17-a092-4301-b5af-6106f3af1b72 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:10.024486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c8a17-a092-4301-b5af-6106f3af1b72 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:10.029088 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906afcbe-36eb-4dfc-bc44-5d961754afa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:10.029741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906afcbe-36eb-4dfc-bc44-5d961754afa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:10.030026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906afcbe-36eb-4dfc-bc44-5d961754afa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:10.030306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906afcbe-36eb-4dfc-bc44-5d961754afa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:10.031068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906afcbe-36eb-4dfc-bc44-5d961754afa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:10.033176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906afcbe-36eb-4dfc-bc44-5d961754afa1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:10.043500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 961/1938] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:05:10.048139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ec0b3-9777-4a85-913e-ae378490b3e8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:10.058353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802ec0b3-9777-4a85-913e-ae378490b3e8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:10.062963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844806e-83f4-41ee-aacb-08a7788bcee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:10.063830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844806e-83f4-41ee-aacb-08a7788bcee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:10.064114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844806e-83f4-41ee-aacb-08a7788bcee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:10.064389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844806e-83f4-41ee-aacb-08a7788bcee7 admin admin] PATH_INFO: `/v3/projects/a962c8c9085844408c9ba1f7cc7fc495/users/c3604b52d0554b179ef7a799dfb80abe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:10.079156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844806e-83f4-41ee-aacb-08a7788bcee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a962c8c9085844408c9ba1f7cc7fc495, user_id=c3604b52d0554b179ef7a799dfb80abe, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:10.082009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844806e-83f4-41ee-aacb-08a7788bcee7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:10.094318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 978/1939] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:10 2022] PUT /identity/v3/projects/a962c8c9085844408c9ba1f7cc7fc495/users/c3604b52d0554b179ef7a799dfb80abe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:10.098832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906afcbe-36eb-4dfc-bc44-5d961754afa1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:10.108694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906afcbe-36eb-4dfc-bc44-5d961754afa1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:10.142751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:10.166435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61d70c-bacc-42e4-9a25-b12d56627529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:10.167125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d70c-bacc-42e4-9a25-b12d56627529 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:10.167405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d70c-bacc-42e4-9a25-b12d56627529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:10.167679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61d70c-bacc-42e4-9a25-b12d56627529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:10.168388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61d70c-bacc-42e4-9a25-b12d56627529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:10.169991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61d70c-bacc-42e4-9a25-b12d56627529 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:10.179326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 962/1940] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:10.183933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844806e-83f4-41ee-aacb-08a7788bcee7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:10.194177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844806e-83f4-41ee-aacb-08a7788bcee7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:10.198765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c237de3-74b9-4e26-b1b7-f2c5aef9ecd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:10.199665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c237de3-74b9-4e26-b1b7-f2c5aef9ecd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:10.199948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c237de3-74b9-4e26-b1b7-f2c5aef9ecd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:10.200222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c237de3-74b9-4e26-b1b7-f2c5aef9ecd3 admin admin] PATH_INFO: `/v3/projects/a962c8c9085844408c9ba1f7cc7fc495/users/c3604b52d0554b179ef7a799dfb80abe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:10.202165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c237de3-74b9-4e26-b1b7-f2c5aef9ecd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a962c8c9085844408c9ba1f7cc7fc495, user_id=c3604b52d0554b179ef7a799dfb80abe, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:10.205526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c237de3-74b9-4e26-b1b7-f2c5aef9ecd3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:10.215421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 979/1941] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:10 2022] PUT /identity/v3/projects/a962c8c9085844408c9ba1f7cc7fc495/users/c3604b52d0554b179ef7a799dfb80abe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:10.227231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5ec69-b0f2-4783-961d-97b8f676b587 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:10.227523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5ec69-b0f2-4783-961d-97b8f676b587 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:10.227805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5ec69-b0f2-4783-961d-97b8f676b587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:10.253225 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-2eb5ec69-b0f2-4783-961d-97b8f676b587 None None] MFA Rules not processed for user `c3604b52d0554b179ef7a799dfb80abe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:10.254345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2eb5ec69-b0f2-4783-961d-97b8f676b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:10.320382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 963/1942] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:10.334114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c237de3-74b9-4e26-b1b7-f2c5aef9ecd3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:10.336112 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1c237de3-74b9-4e26-b1b7-f2c5aef9ecd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:10.409305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c237de3-74b9-4e26-b1b7-f2c5aef9ecd3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:10.412959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f479d-c37f-40d6-aa6d-f9841baa4694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:10.413603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f479d-c37f-40d6-aa6d-f9841baa4694 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:10.413917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f479d-c37f-40d6-aa6d-f9841baa4694 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:10.414195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f479d-c37f-40d6-aa6d-f9841baa4694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:10.421079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:10.423444 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f479d-c37f-40d6-aa6d-f9841baa4694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:10.425333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f479d-c37f-40d6-aa6d-f9841baa4694 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:10.435801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 980/1943] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:10.522846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5ec69-b0f2-4783-961d-97b8f676b587 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:10.524420 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2eb5ec69-b0f2-4783-961d-97b8f676b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:10.538141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5ec69-b0f2-4783-961d-97b8f676b587 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:10.541902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0e829-451e-4db3-8318-b0866a2d658a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:10.542659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0e829-451e-4db3-8318-b0866a2d658a service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:10.543012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0e829-451e-4db3-8318-b0866a2d658a service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:10.543358 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0e829-451e-4db3-8318-b0866a2d658a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:10.609024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0e829-451e-4db3-8318-b0866a2d658a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:10.610546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0e829-451e-4db3-8318-b0866a2d658a service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:10.619759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 964/1944] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:10.907380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f479d-c37f-40d6-aa6d-f9841baa4694 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:10.969652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f479d-c37f-40d6-aa6d-f9841baa4694 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:10.974167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:10.975084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:10.975372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:10.975651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] PATH_INFO: `/v3/users/6e10b02ca643416fa138141d32b96091` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:10.976843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e10b02ca643416fa138141d32b96091)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:10.978833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:11.033641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e10b02ca643416fa138141d32b96091'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.047767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e10b02ca643416fa138141d32b96091'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.055619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e10b02ca643416fa138141d32b96091'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.056428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e10b02ca643416fa138141d32b96091'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.064016 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e10b02ca643416fa138141d32b96091'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.071937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 981/1945] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:10 2022] DELETE /identity/v3/users/6e10b02ca643416fa138141d32b96091 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:11.076406 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50c381-5b3e-45d8-95eb-e7f37a67a600 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:11.076863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50c381-5b3e-45d8-95eb-e7f37a67a600 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:11.077003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50c381-5b3e-45d8-95eb-e7f37a67a600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:11.095462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7b50c381-5b3e-45d8-95eb-e7f37a67a600 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:11.096669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7b50c381-5b3e-45d8-95eb-e7f37a67a600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:11.146843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:11.156275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 965/1946] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:11.166596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:11.168013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:11.223728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0372ad7-e92b-44e4-b7f7-fc475518697e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:11.227348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3897157-619e-43b3-9b2e-c68af4536ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:11.227983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3897157-619e-43b3-9b2e-c68af4536ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:11.228264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3897157-619e-43b3-9b2e-c68af4536ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:11.228536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3897157-619e-43b3-9b2e-c68af4536ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:11.237303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3897157-619e-43b3-9b2e-c68af4536ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:11.238742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3897157-619e-43b3-9b2e-c68af4536ea4 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:11.247967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 982/1947] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:11.643855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50c381-5b3e-45d8-95eb-e7f37a67a600 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:11.645682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7b50c381-5b3e-45d8-95eb-e7f37a67a600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:11.657843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50c381-5b3e-45d8-95eb-e7f37a67a600 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:11.662548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:11.663418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:11.663697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:11.663995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] PATH_INFO: `/v3/projects/0f2612edd696457597e8719fc18383ba` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:11.665153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f2612edd696457597e8719fc18383ba)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:11.667186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:11.720853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f2612edd696457597e8719fc18383ba', 'request_id': 'req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.726353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f2612edd696457597e8719fc18383ba', 'request_id': 'req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.732479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f2612edd696457597e8719fc18383ba', 'request_id': 'req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.745619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] The token cache is being invalidate because project 0f2612edd696457597e8719fc18383ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:11.745911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.746577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.748342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 966/1948] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:11 2022] DELETE /identity/v3/projects/0f2612edd696457597e8719fc18383ba => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:11.752935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3897157-619e-43b3-9b2e-c68af4536ea4 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:11.762725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3897157-619e-43b3-9b2e-c68af4536ea4 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:11.775648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:11.837006 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:11.839193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:11.839946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:11.840365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] PATH_INFO: `/v3/users/c3604b52d0554b179ef7a799dfb80abe` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:11.841682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3604b52d0554b179ef7a799dfb80abe)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:11.843944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:11.894071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3604b52d0554b179ef7a799dfb80abe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.906148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3604b52d0554b179ef7a799dfb80abe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.913786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3604b52d0554b179ef7a799dfb80abe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.914428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3604b52d0554b179ef7a799dfb80abe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.921458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a71e5766-9357-4fd5-b0df-c1c76c12d5fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3604b52d0554b179ef7a799dfb80abe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:11.928404 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 983/1949] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:11 2022] DELETE /identity/v3/users/c3604b52d0554b179ef7a799dfb80abe => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:12.212545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:12.214030 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:12.228799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2e49-8577-42ec-b6d7-5dc0d04f49f4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:12.289503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:12.290375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:12.290693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:12.290975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] PATH_INFO: `/v3/projects/a962c8c9085844408c9ba1f7cc7fc495` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:12.292161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a962c8c9085844408c9ba1f7cc7fc495)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:12.294114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:12.351657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a962c8c9085844408c9ba1f7cc7fc495', 'request_id': 'req-61b53daa-c4d6-420e-a770-b0110724eafc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:12.358269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a962c8c9085844408c9ba1f7cc7fc495', 'request_id': 'req-61b53daa-c4d6-420e-a770-b0110724eafc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:12.365293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a962c8c9085844408c9ba1f7cc7fc495', 'request_id': 'req-61b53daa-c4d6-420e-a770-b0110724eafc'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:12.378964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] The token cache is being invalidate because project a962c8c9085844408c9ba1f7cc7fc495 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:12.379250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:12.379888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-61b53daa-c4d6-420e-a770-b0110724eafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:12.381555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 967/1950] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:12 2022] DELETE /identity/v3/projects/a962c8c9085844408c9ba1f7cc7fc495 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:12.387363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ba991-30d9-497c-b332-15dfa7002bba None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:12.387869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ba991-30d9-497c-b332-15dfa7002bba None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:12.387968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ba991-30d9-497c-b332-15dfa7002bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:12.407559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cf5ba991-30d9-497c-b332-15dfa7002bba None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:12.408763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cf5ba991-30d9-497c-b332-15dfa7002bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:12.479808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 984/1951] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:12.487596 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b66403-b196-4eeb-a8c2-d6c7a889e04c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:12.487889 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b66403-b196-4eeb-a8c2-d6c7a889e04c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:12.488172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b66403-b196-4eeb-a8c2-d6c7a889e04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:12.506375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-63b66403-b196-4eeb-a8c2-d6c7a889e04c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:12.507613 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-63b66403-b196-4eeb-a8c2-d6c7a889e04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:12.515463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 968/1952] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:12.520095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ba991-30d9-497c-b332-15dfa7002bba None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:12.530260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ba991-30d9-497c-b332-15dfa7002bba None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:12.535699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbfb98-5d65-41cb-b600-ba1f22809b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:12.536388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbfb98-5d65-41cb-b600-ba1f22809b50 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:12.536672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbfb98-5d65-41cb-b600-ba1f22809b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:12.536951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbfb98-5d65-41cb-b600-ba1f22809b50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:12.537673 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbfb98-5d65-41cb-b600-ba1f22809b50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:12.537980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbfb98-5d65-41cb-b600-ba1f22809b50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:12.539637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbfb98-5d65-41cb-b600-ba1f22809b50 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:12.550850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 985/1953] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:05:12.555052 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b66403-b196-4eeb-a8c2-d6c7a889e04c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:12.560646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:12.564299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b66403-b196-4eeb-a8c2-d6c7a889e04c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:12.570503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18909f7-a95c-4b69-ad5e-9c7dfb8838a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:12.571329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18909f7-a95c-4b69-ad5e-9c7dfb8838a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:12.571790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18909f7-a95c-4b69-ad5e-9c7dfb8838a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:12.572264 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18909f7-a95c-4b69-ad5e-9c7dfb8838a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:12.572974 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18909f7-a95c-4b69-ad5e-9c7dfb8838a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:12.574916 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18909f7-a95c-4b69-ad5e-9c7dfb8838a0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:12.601054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 969/1954] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:12 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:12.605131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbfb98-5d65-41cb-b600-ba1f22809b50 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:12.616022 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbfb98-5d65-41cb-b600-ba1f22809b50 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:12.641884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:12.682928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4a129-7b5c-4273-91df-fc5061fdc464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:12.683627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4a129-7b5c-4273-91df-fc5061fdc464 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:12.683942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4a129-7b5c-4273-91df-fc5061fdc464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:12.684253 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4a129-7b5c-4273-91df-fc5061fdc464 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:12.685133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4a129-7b5c-4273-91df-fc5061fdc464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:12.687299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4a129-7b5c-4273-91df-fc5061fdc464 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:12.713437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 986/1955] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:12 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:12.717682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18909f7-a95c-4b69-ad5e-9c7dfb8838a0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:12.730269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18909f7-a95c-4b69-ad5e-9c7dfb8838a0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:12.736502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e3812-5d57-4541-bfe9-a303b06ed8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:12.737173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e3812-5d57-4541-bfe9-a303b06ed8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:12.737456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e3812-5d57-4541-bfe9-a303b06ed8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:12.737735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e3812-5d57-4541-bfe9-a303b06ed8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:12.738481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e3812-5d57-4541-bfe9-a303b06ed8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:12.740318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e3812-5d57-4541-bfe9-a303b06ed8e1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:12.754569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 970/1956] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:12.758916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4a129-7b5c-4273-91df-fc5061fdc464 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:12.770825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4a129-7b5c-4273-91df-fc5061fdc464 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:12.776105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f035477-8366-44f7-b47c-e2d0f0f7bc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:12.777079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035477-8366-44f7-b47c-e2d0f0f7bc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:12.777454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035477-8366-44f7-b47c-e2d0f0f7bc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:12.777742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f035477-8366-44f7-b47c-e2d0f0f7bc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:12.778557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f035477-8366-44f7-b47c-e2d0f0f7bc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:12.780191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f035477-8366-44f7-b47c-e2d0f0f7bc8c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:12.792367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 987/1957] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:05:12.796945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e3812-5d57-4541-bfe9-a303b06ed8e1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:12.812082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e3812-5d57-4541-bfe9-a303b06ed8e1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:12.817530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916c323-3879-44f1-9028-c7a1a0b180ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:12.818626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916c323-3879-44f1-9028-c7a1a0b180ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:12.818906 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916c323-3879-44f1-9028-c7a1a0b180ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:12.819181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916c323-3879-44f1-9028-c7a1a0b180ce admin admin] PATH_INFO: `/v3/projects/971dd1a0b88e4fa9ba667db8983a127e/users/4243daae2d1243b59e179b3e2c0ca1d7/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:12.835181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916c323-3879-44f1-9028-c7a1a0b180ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=971dd1a0b88e4fa9ba667db8983a127e, user_id=4243daae2d1243b59e179b3e2c0ca1d7, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:12.837844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7916c323-3879-44f1-9028-c7a1a0b180ce admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:12.851397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 971/1958] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:12 2022] PUT /identity/v3/projects/971dd1a0b88e4fa9ba667db8983a127e/users/4243daae2d1243b59e179b3e2c0ca1d7/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:12.865342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6edf1e-5803-422a-ab79-66d9d1b13fc9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:12.865649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6edf1e-5803-422a-ab79-66d9d1b13fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:12.865943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6edf1e-5803-422a-ab79-66d9d1b13fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:12.903038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3f6edf1e-5803-422a-ab79-66d9d1b13fc9 None None] MFA Rules not processed for user `4243daae2d1243b59e179b3e2c0ca1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:12.904355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3f6edf1e-5803-422a-ab79-66d9d1b13fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:12.937026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916c323-3879-44f1-9028-c7a1a0b180ce admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:12.940571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7916c323-3879-44f1-9028-c7a1a0b180ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:13.061423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 988/1959] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:13.063525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6edf1e-5803-422a-ab79-66d9d1b13fc9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:13.111878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7916c323-3879-44f1-9028-c7a1a0b180ce admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:13.135554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:13.136809 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.137109 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.137397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] PATH_INFO: `/v3/users/f86be24085e243ed98dbb195675a5d69` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.140797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86be24085e243ed98dbb195675a5d69)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:13.142904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:13.218474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86be24085e243ed98dbb195675a5d69'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:13.219661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86be24085e243ed98dbb195675a5d69'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:13.229912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6edf1e-5803-422a-ab79-66d9d1b13fc9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:13.232411 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86be24085e243ed98dbb195675a5d69'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:13.233378 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86be24085e243ed98dbb195675a5d69'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:13.243338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dc7c3-fa63-4da1-91f7-4665e13e7a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:13.244022 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dc7c3-fa63-4da1-91f7-4665e13e7a4f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.244307 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dc7c3-fa63-4da1-91f7-4665e13e7a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.244584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dc7c3-fa63-4da1-91f7-4665e13e7a4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.245383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772dc7c3-fa63-4da1-91f7-4665e13e7a4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:13.247359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772dc7c3-fa63-4da1-91f7-4665e13e7a4f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:13.259907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-2a1ca964-495c-43e7-9451-4260cc7f0162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86be24085e243ed98dbb195675a5d69'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:13.281133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 972/1960] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:12 2022] DELETE /identity/v3/users/f86be24085e243ed98dbb195675a5d69 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:13.287148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdddc10-0413-4a46-b809-58d167e66141 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.287673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdddc10-0413-4a46-b809-58d167e66141 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.288104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdddc10-0413-4a46-b809-58d167e66141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.291242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 989/1961] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:13 2022] POST /identity/v3/projects => generated 383 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:13.296150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dc7c3-fa63-4da1-91f7-4665e13e7a4f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:13.300542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-772dc7c3-fa63-4da1-91f7-4665e13e7a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:13.327219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:13.327321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ebdddc10-0413-4a46-b809-58d167e66141 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:13.328321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ebdddc10-0413-4a46-b809-58d167e66141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:13.355128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:13.392501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dc7c3-fa63-4da1-91f7-4665e13e7a4f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:13.397913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfae5d-ebfb-44f1-b341-5259839b7a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:13.398359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfae5d-ebfb-44f1-b341-5259839b7a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.398683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfae5d-ebfb-44f1-b341-5259839b7a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.398973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfae5d-ebfb-44f1-b341-5259839b7a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.400537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfae5d-ebfb-44f1-b341-5259839b7a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:13.401639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfae5d-ebfb-44f1-b341-5259839b7a31 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:13.423789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 973/1962] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:13.427642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 990/1963] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:13 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:13.429473 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdddc10-0413-4a46-b809-58d167e66141 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:13.433091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfae5d-ebfb-44f1-b341-5259839b7a31 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:13.434970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-90cfae5d-ebfb-44f1-b341-5259839b7a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:13.444132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdddc10-0413-4a46-b809-58d167e66141 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:13.449996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aac8a-b192-41c5-b674-e547be7f096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:13.450928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aac8a-b192-41c5-b674-e547be7f096c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.451373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aac8a-b192-41c5-b674-e547be7f096c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.451657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8aac8a-b192-41c5-b674-e547be7f096c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.452573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aac8a-b192-41c5-b674-e547be7f096c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:13.454301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8aac8a-b192-41c5-b674-e547be7f096c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:13.470361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 974/1964] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:13.471544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aac8a-b192-41c5-b674-e547be7f096c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:13.486525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8aac8a-b192-41c5-b674-e547be7f096c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:13.496486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c992f-897e-4ce2-9566-932bd917d4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:13.496486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c992f-897e-4ce2-9566-932bd917d4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.496486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c992f-897e-4ce2-9566-932bd917d4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.496486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c992f-897e-4ce2-9566-932bd917d4c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.496843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c992f-897e-4ce2-9566-932bd917d4c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:13.496843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c992f-897e-4ce2-9566-932bd917d4c8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:13.504144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfae5d-ebfb-44f1-b341-5259839b7a31 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:13.507647 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 975/1965] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:13.511400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add302b-2275-4039-a715-7c764e8db20f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:13.511845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c992f-897e-4ce2-9566-932bd917d4c8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:13.512233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add302b-2275-4039-a715-7c764e8db20f service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.512516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add302b-2275-4039-a715-7c764e8db20f service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.512792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add302b-2275-4039-a715-7c764e8db20f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.522310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c992f-897e-4ce2-9566-932bd917d4c8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:13.523072 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add302b-2275-4039-a715-7c764e8db20f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:13.524496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add302b-2275-4039-a715-7c764e8db20f service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:13.528635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42465bd4-512b-4b4f-b396-4bd527aade57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:13.530091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42465bd4-512b-4b4f-b396-4bd527aade57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.531169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42465bd4-512b-4b4f-b396-4bd527aade57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.531564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42465bd4-512b-4b4f-b396-4bd527aade57 admin admin] PATH_INFO: `/v3/projects/ab6ad362411042b0b592d85a1d684aee/users/807255e520214fa9a2e4c05d6400dfa3/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.536184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 991/1966] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:13.550504 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42465bd4-512b-4b4f-b396-4bd527aade57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6ad362411042b0b592d85a1d684aee, user_id=807255e520214fa9a2e4c05d6400dfa3, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:13.553151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42465bd4-512b-4b4f-b396-4bd527aade57 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:13.566480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 976/1967] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:13 2022] PUT /identity/v3/projects/ab6ad362411042b0b592d85a1d684aee/users/807255e520214fa9a2e4c05d6400dfa3/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:13.576471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabc7dc-878a-4853-b1e2-1116874a1e1b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.576769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabc7dc-878a-4853-b1e2-1116874a1e1b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.577056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabc7dc-878a-4853-b1e2-1116874a1e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.604751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5fabc7dc-878a-4853-b1e2-1116874a1e1b None None] MFA Rules not processed for user `807255e520214fa9a2e4c05d6400dfa3`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:13.606015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5fabc7dc-878a-4853-b1e2-1116874a1e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:13.682632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 992/1968] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:13.687806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42465bd4-512b-4b4f-b396-4bd527aade57 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:13.796872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42465bd4-512b-4b4f-b396-4bd527aade57 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:13.815983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1dfcf-3e1f-4cc0-89e3-8d36038e3230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:13.816948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1dfcf-3e1f-4cc0-89e3-8d36038e3230 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.817584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1dfcf-3e1f-4cc0-89e3-8d36038e3230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.818063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1dfcf-3e1f-4cc0-89e3-8d36038e3230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.819345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1dfcf-3e1f-4cc0-89e3-8d36038e3230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:13.821563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1dfcf-3e1f-4cc0-89e3-8d36038e3230 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:13.851995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 977/1969] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:13 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:13.856731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabc7dc-878a-4853-b1e2-1116874a1e1b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:13.870047 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabc7dc-878a-4853-b1e2-1116874a1e1b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:13.905977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1dfcf-3e1f-4cc0-89e3-8d36038e3230 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:13.908633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7bb1dfcf-3e1f-4cc0-89e3-8d36038e3230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:13.944465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:13.974485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eca473-dd0e-4d32-8f63-bbf36e149f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:13.974485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eca473-dd0e-4d32-8f63-bbf36e149f65 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.974485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eca473-dd0e-4d32-8f63-bbf36e149f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.974485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eca473-dd0e-4d32-8f63-bbf36e149f65 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.974854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eca473-dd0e-4d32-8f63-bbf36e149f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:13.974854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eca473-dd0e-4d32-8f63-bbf36e149f65 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:13.987719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1dfcf-3e1f-4cc0-89e3-8d36038e3230 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:13.995274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:13.996387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:13.996977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:13.997435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] PATH_INFO: `/v3/projects/491c4328c56a4565abc99f963d7a785c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:13.999471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=491c4328c56a4565abc99f963d7a785c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:14.001028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 993/1970] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:13 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:14.001780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:14.005247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eca473-dd0e-4d32-8f63-bbf36e149f65 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:14.010990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:14.017966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eca473-dd0e-4d32-8f63-bbf36e149f65 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:14.029039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55c717-4811-4f96-a009-3c8f988dfa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:14.029039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55c717-4811-4f96-a009-3c8f988dfa26 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:14.029039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55c717-4811-4f96-a009-3c8f988dfa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:14.029039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55c717-4811-4f96-a009-3c8f988dfa26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:14.029516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55c717-4811-4f96-a009-3c8f988dfa26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:14.029516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55c717-4811-4f96-a009-3c8f988dfa26 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:14.042958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 994/1971] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:14.046963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55c717-4811-4f96-a009-3c8f988dfa26 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:14.056962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55c717-4811-4f96-a009-3c8f988dfa26 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:14.064719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '491c4328c56a4565abc99f963d7a785c', 'request_id': 'req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:14.071381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '491c4328c56a4565abc99f963d7a785c', 'request_id': 'req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:14.079960 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '491c4328c56a4565abc99f963d7a785c', 'request_id': 'req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:14.093668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] The token cache is being invalidate because project 491c4328c56a4565abc99f963d7a785c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:14.093960 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:14.095350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:14.097281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 978/1972] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:13 2022] DELETE /identity/v3/projects/491c4328c56a4565abc99f963d7a785c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:14.101369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:14.102806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:14.124258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ab996-39de-4a02-a5ec-e5aeca13953e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:14.125494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab996-39de-4a02-a5ec-e5aeca13953e admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:14.125858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab996-39de-4a02-a5ec-e5aeca13953e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:14.126296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ab996-39de-4a02-a5ec-e5aeca13953e admin admin] PATH_INFO: `/v3/projects/0779edfd81be48a690675589940b1558/users/5a833289bea84f12aed5e0c42a610535/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:14.143805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ab996-39de-4a02-a5ec-e5aeca13953e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0779edfd81be48a690675589940b1558, user_id=5a833289bea84f12aed5e0c42a610535, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:14.147570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ab996-39de-4a02-a5ec-e5aeca13953e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:14.167890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 995/1973] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:14 2022] PUT /identity/v3/projects/0779edfd81be48a690675589940b1558/users/5a833289bea84f12aed5e0c42a610535/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:14.171796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ab996-39de-4a02-a5ec-e5aeca13953e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:14.173550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-122ab996-39de-4a02-a5ec-e5aeca13953e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:14.187475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8bb3a-3622-4cc5-ad61-10773ad6a328 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:14.272722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:14.273676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:14.273965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:14.274293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] PATH_INFO: `/v3/users/4936be9aa5dd44b2a64761aec1fe100c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:14.276039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4936be9aa5dd44b2a64761aec1fe100c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:14.280258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:14.333509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ab996-39de-4a02-a5ec-e5aeca13953e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:14.340414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef88d64-ffcd-41b6-bbcf-a37f8d46e188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:14.341618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef88d64-ffcd-41b6-bbcf-a37f8d46e188 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:14.342037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef88d64-ffcd-41b6-bbcf-a37f8d46e188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:14.342423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef88d64-ffcd-41b6-bbcf-a37f8d46e188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:14.343310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef88d64-ffcd-41b6-bbcf-a37f8d46e188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:14.345309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef88d64-ffcd-41b6-bbcf-a37f8d46e188 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:14.347202 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:14.352287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4936be9aa5dd44b2a64761aec1fe100c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:14.354473 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4936be9aa5dd44b2a64761aec1fe100c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:14.364111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 996/1974] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:05:14.369175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef88d64-ffcd-41b6-bbcf-a37f8d46e188 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:14.371020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4936be9aa5dd44b2a64761aec1fe100c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:14.371781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4936be9aa5dd44b2a64761aec1fe100c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:14.371896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2ef88d64-ffcd-41b6-bbcf-a37f8d46e188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:14.388263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef88d64-ffcd-41b6-bbcf-a37f8d46e188 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:14.390601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1d0c8b4b-d4bf-46ef-bd54-81ef590df41b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4936be9aa5dd44b2a64761aec1fe100c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:14.396114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c269e-876b-4d6c-8ba1-427ada542811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:14.397442 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c269e-876b-4d6c-8ba1-427ada542811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:14.397843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c269e-876b-4d6c-8ba1-427ada542811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:14.398702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c269e-876b-4d6c-8ba1-427ada542811 admin admin] PATH_INFO: `/v3/projects/0779edfd81be48a690675589940b1558/users/5a833289bea84f12aed5e0c42a610535/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:14.400995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c269e-876b-4d6c-8ba1-427ada542811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0779edfd81be48a690675589940b1558, user_id=5a833289bea84f12aed5e0c42a610535, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:14.404090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c269e-876b-4d6c-8ba1-427ada542811 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:14.412082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 979/1975] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:14 2022] DELETE /identity/v3/users/4936be9aa5dd44b2a64761aec1fe100c => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:14.425743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 997/1976] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:14 2022] PUT /identity/v3/projects/0779edfd81be48a690675589940b1558/users/5a833289bea84f12aed5e0c42a610535/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:14.437304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0546e26-023d-45f3-a36f-1a4aae43162e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:14.438513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0546e26-023d-45f3-a36f-1a4aae43162e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:14.439330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0546e26-023d-45f3-a36f-1a4aae43162e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:14.482046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e0546e26-023d-45f3-a36f-1a4aae43162e None None] MFA Rules not processed for user `5a833289bea84f12aed5e0c42a610535`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:14.484107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e0546e26-023d-45f3-a36f-1a4aae43162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:14.643912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 980/1977] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:14.654485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c269e-876b-4d6c-8ba1-427ada542811 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:14.657457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-168c269e-876b-4d6c-8ba1-427ada542811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:14.716530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:14.786830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c269e-876b-4d6c-8ba1-427ada542811 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:14.884667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffc411-098f-4f50-b2db-15fd1247c4c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:14.885336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffc411-098f-4f50-b2db-15fd1247c4c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:14.886199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffc411-098f-4f50-b2db-15fd1247c4c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:14.900408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffc411-098f-4f50-b2db-15fd1247c4c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:14.907016 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5fffc411-098f-4f50-b2db-15fd1247c4c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:15.009594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffc411-098f-4f50-b2db-15fd1247c4c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:15.011806 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffc411-098f-4f50-b2db-15fd1247c4c1 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:15.024079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 998/1978] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:15.218648 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0546e26-023d-45f3-a36f-1a4aae43162e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:15.221168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e0546e26-023d-45f3-a36f-1a4aae43162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:15.394385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0546e26-023d-45f3-a36f-1a4aae43162e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:15.404736 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:15.405623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:15.405908 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:15.406184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] PATH_INFO: `/v3/projects/c5388488bd054f7faa4c8c16adf19876` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:15.409631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5388488bd054f7faa4c8c16adf19876)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:15.413189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:15.551682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5388488bd054f7faa4c8c16adf19876', 'request_id': 'req-69d230de-f82f-4487-a586-cca6fa2959ea'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:15.553110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:15.560906 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5388488bd054f7faa4c8c16adf19876', 'request_id': 'req-69d230de-f82f-4487-a586-cca6fa2959ea'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:15.569011 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5388488bd054f7faa4c8c16adf19876', 'request_id': 'req-69d230de-f82f-4487-a586-cca6fa2959ea'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:15.603931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] The token cache is being invalidate because project c5388488bd054f7faa4c8c16adf19876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:15.604522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:15.605380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:15.610715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 981/1979] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:15 2022] DELETE /identity/v3/projects/c5388488bd054f7faa4c8c16adf19876 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:15.614386 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffc411-098f-4f50-b2db-15fd1247c4c1 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:15.632545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffc411-098f-4f50-b2db-15fd1247c4c1 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:15.639561 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:15.643596 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:15.788616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d230de-f82f-4487-a586-cca6fa2959ea admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:15.796225 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd771218-841d-4220-a65f-69deef7e7ada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:15.798056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771218-841d-4220-a65f-69deef7e7ada service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:15.798854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:15.799273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771218-841d-4220-a65f-69deef7e7ada service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:15.799750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:15.800090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:15.800380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] PATH_INFO: `/v3/users/f9c1aee0f94a414d9604fe3832962102` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:15.800929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd771218-841d-4220-a65f-69deef7e7ada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:15.801713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c1aee0f94a414d9604fe3832962102)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:15.803357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dd771218-841d-4220-a65f-69deef7e7ada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:15.806347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:15.858788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:15.870784 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c1aee0f94a414d9604fe3832962102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:15.895026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c1aee0f94a414d9604fe3832962102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:15.905705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd771218-841d-4220-a65f-69deef7e7ada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:15.908079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd771218-841d-4220-a65f-69deef7e7ada service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:15.912535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c1aee0f94a414d9604fe3832962102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:15.914038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c1aee0f94a414d9604fe3832962102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:15.920571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 982/1980] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:15.924978 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0d2b46b-472e-4700-8510-31fe57aff721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c1aee0f94a414d9604fe3832962102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:15.939490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 999/1981] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:15 2022] DELETE /identity/v3/users/f9c1aee0f94a414d9604fe3832962102 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:16.327128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd771218-841d-4220-a65f-69deef7e7ada service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:16.329121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dd771218-841d-4220-a65f-69deef7e7ada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:16.355027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd771218-841d-4220-a65f-69deef7e7ada service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:16.444342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:16.445430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:16.446076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:16.446076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] PATH_INFO: `/v3/projects/f65c76fdd33845fab0d6540ec33edf6f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:16.447493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f65c76fdd33845fab0d6540ec33edf6f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:16.451097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:16.529422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:16.540553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f65c76fdd33845fab0d6540ec33edf6f', 'request_id': 'req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:16.548775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f65c76fdd33845fab0d6540ec33edf6f', 'request_id': 'req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:16.559030 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f65c76fdd33845fab0d6540ec33edf6f', 'request_id': 'req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:16.583624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] The token cache is being invalidate because project f65c76fdd33845fab0d6540ec33edf6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:16.583624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:16.583624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-b7524f83-4ce3-479a-a838-bc2e4b8cdb19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:16.583624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 983/1982] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:16 2022] DELETE /identity/v3/projects/f65c76fdd33845fab0d6540ec33edf6f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:16.587647 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f0a0d-eaa4-4b43-99a0-41ada6889bc6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:16.588047 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f0a0d-eaa4-4b43-99a0-41ada6889bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:16.589154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f0a0d-eaa4-4b43-99a0-41ada6889bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:16.609379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f75f0a0d-eaa4-4b43-99a0-41ada6889bc6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:16.611430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f75f0a0d-eaa4-4b43-99a0-41ada6889bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:16.683745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1000/1983] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:16.692775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708ac22-10d0-483d-9dcd-e71884f5e38c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:16.693914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708ac22-10d0-483d-9dcd-e71884f5e38c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:16.693914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708ac22-10d0-483d-9dcd-e71884f5e38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:16.718070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0708ac22-10d0-483d-9dcd-e71884f5e38c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:16.720004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0708ac22-10d0-483d-9dcd-e71884f5e38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:16.730148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 984/1984] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:16.732748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f0a0d-eaa4-4b43-99a0-41ada6889bc6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:16.745931 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f0a0d-eaa4-4b43-99a0-41ada6889bc6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:16.757615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b065502-7693-4051-9101-9d13af14e98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:16.758863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b065502-7693-4051-9101-9d13af14e98a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:16.759118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b065502-7693-4051-9101-9d13af14e98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:16.759404 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b065502-7693-4051-9101-9d13af14e98a admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:16.760430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b065502-7693-4051-9101-9d13af14e98a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:16.760745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b065502-7693-4051-9101-9d13af14e98a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:16.762612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b065502-7693-4051-9101-9d13af14e98a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:16.782011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1001/1985] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:05:16.791900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708ac22-10d0-483d-9dcd-e71884f5e38c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:16.803302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708ac22-10d0-483d-9dcd-e71884f5e38c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:16.818165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321382ac-2393-4987-aa6c-a77b91fef5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:16.819064 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321382ac-2393-4987-aa6c-a77b91fef5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:16.819374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321382ac-2393-4987-aa6c-a77b91fef5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:16.819661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321382ac-2393-4987-aa6c-a77b91fef5e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:16.820381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321382ac-2393-4987-aa6c-a77b91fef5e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:16.823799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321382ac-2393-4987-aa6c-a77b91fef5e3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:16.856187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 985/1986] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:16 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:16.861016 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b065502-7693-4051-9101-9d13af14e98a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:16.872678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b065502-7693-4051-9101-9d13af14e98a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:16.971643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9251ee-5992-418d-ae16-68d899251ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:16.972684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9251ee-5992-418d-ae16-68d899251ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:16.973163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9251ee-5992-418d-ae16-68d899251ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:16.973747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9251ee-5992-418d-ae16-68d899251ac6 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:16.975187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9251ee-5992-418d-ae16-68d899251ac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:16.978595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9251ee-5992-418d-ae16-68d899251ac6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:16.999604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321382ac-2393-4987-aa6c-a77b91fef5e3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:17.001632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-321382ac-2393-4987-aa6c-a77b91fef5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:17.022780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1002/1987] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:16 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:17.023502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9251ee-5992-418d-ae16-68d899251ac6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:17.042953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9251ee-5992-418d-ae16-68d899251ac6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:17.051830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961a20c-1092-4103-a575-c357cf6970eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:17.052528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961a20c-1092-4103-a575-c357cf6970eb admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:17.052813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961a20c-1092-4103-a575-c357cf6970eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:17.053091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961a20c-1092-4103-a575-c357cf6970eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:17.054263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961a20c-1092-4103-a575-c357cf6970eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:17.055554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961a20c-1092-4103-a575-c357cf6970eb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:17.077119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1003/1988] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:17.082083 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961a20c-1092-4103-a575-c357cf6970eb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:17.104637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961a20c-1092-4103-a575-c357cf6970eb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:17.110836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2604a-7d3e-4e42-848b-19cf74604730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:17.111503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2604a-7d3e-4e42-848b-19cf74604730 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:17.111796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2604a-7d3e-4e42-848b-19cf74604730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:17.112077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2604a-7d3e-4e42-848b-19cf74604730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:17.112894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2604a-7d3e-4e42-848b-19cf74604730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:17.114428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2604a-7d3e-4e42-848b-19cf74604730 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:17.116675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321382ac-2393-4987-aa6c-a77b91fef5e3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:17.121930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef91e8a-3756-4e91-91cd-ce5d1df3b10a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:17.122624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef91e8a-3756-4e91-91cd-ce5d1df3b10a service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:17.124071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef91e8a-3756-4e91-91cd-ce5d1df3b10a service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:17.124356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef91e8a-3756-4e91-91cd-ce5d1df3b10a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:17.127309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1004/1989] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:17.132031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2604a-7d3e-4e42-848b-19cf74604730 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:17.135988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-aef91e8a-3756-4e91-91cd-ce5d1df3b10a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:17.144255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2604a-7d3e-4e42-848b-19cf74604730 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:17.151884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d80a5-73a7-4fa9-a5bd-27d52b4e1b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:17.153212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d80a5-73a7-4fa9-a5bd-27d52b4e1b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:17.153749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d80a5-73a7-4fa9-a5bd-27d52b4e1b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:17.154374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d80a5-73a7-4fa9-a5bd-27d52b4e1b19 admin admin] PATH_INFO: `/v3/projects/9a6a63e6c17d4b8bbd698f6e652de935/users/4a3f5eeb2f01498aa4c84c5ecc6a6cbf/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:17.159276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:17.180857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d80a5-73a7-4fa9-a5bd-27d52b4e1b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6a63e6c17d4b8bbd698f6e652de935, user_id=4a3f5eeb2f01498aa4c84c5ecc6a6cbf, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:17.185677 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d80a5-73a7-4fa9-a5bd-27d52b4e1b19 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:17.210143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1005/1990] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:17 2022] PUT /identity/v3/projects/9a6a63e6c17d4b8bbd698f6e652de935/users/4a3f5eeb2f01498aa4c84c5ecc6a6cbf/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:17.210143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bc5ae-836e-4b29-bfa2-6e2988f02173 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:17.210143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bc5ae-836e-4b29-bfa2-6e2988f02173 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:17.210143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bc5ae-836e-4b29-bfa2-6e2988f02173 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:17.210832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1006/1991] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:05:17.211982 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf2f57-7dc1-47af-898a-f75bc9994ff4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:17.212285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf2f57-7dc1-47af-898a-f75bc9994ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:17.212567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf2f57-7dc1-47af-898a-f75bc9994ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:17.223428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef91e8a-3756-4e91-91cd-ce5d1df3b10a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:17.225090 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef91e8a-3756-4e91-91cd-ce5d1df3b10a service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:17.233157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-62cf2f57-7dc1-47af-898a-f75bc9994ff4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:17.234501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-62cf2f57-7dc1-47af-898a-f75bc9994ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:17.236802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 986/1992] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:17.238166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f81a73-4bd9-4b7c-8617-57fa7658e7fe None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:17.238542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f81a73-4bd9-4b7c-8617-57fa7658e7fe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:17.238764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f81a73-4bd9-4b7c-8617-57fa7658e7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:17.265769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b4f81a73-4bd9-4b7c-8617-57fa7658e7fe None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:17.268108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b4f81a73-4bd9-4b7c-8617-57fa7658e7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:17.327646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:17.347092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1007/1993] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:17.355297 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf2f57-7dc1-47af-898a-f75bc9994ff4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:17.357736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-62cf2f57-7dc1-47af-898a-f75bc9994ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:17.373632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 987/1994] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:05:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:17.382981 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f81a73-4bd9-4b7c-8617-57fa7658e7fe None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:17.384581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b4f81a73-4bd9-4b7c-8617-57fa7658e7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:17.400598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f81a73-4bd9-4b7c-8617-57fa7658e7fe None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:17.407347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960548e-d6db-46e8-a816-6134b9acef7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:17.408128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960548e-d6db-46e8-a816-6134b9acef7b service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:17.408519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960548e-d6db-46e8-a816-6134b9acef7b service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:17.410197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960548e-d6db-46e8-a816-6134b9acef7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:17.422667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960548e-d6db-46e8-a816-6134b9acef7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:17.424927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960548e-d6db-46e8-a816-6134b9acef7b service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:17.436403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 988/1995] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:17.437664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf2f57-7dc1-47af-898a-f75bc9994ff4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:17.442315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278efda-d4a1-4ffb-a0d2-6cb218bbeb41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:17.443013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278efda-d4a1-4ffb-a0d2-6cb218bbeb41 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:17.443305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278efda-d4a1-4ffb-a0d2-6cb218bbeb41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:17.443753 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278efda-d4a1-4ffb-a0d2-6cb218bbeb41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:17.457620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278efda-d4a1-4ffb-a0d2-6cb218bbeb41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:17.459679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278efda-d4a1-4ffb-a0d2-6cb218bbeb41 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:17.471473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1008/1996] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:18.295439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960548e-d6db-46e8-a816-6134b9acef7b service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:18.369967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960548e-d6db-46e8-a816-6134b9acef7b service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:18.378032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4d835-f98f-4776-a606-5cf910e3c769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:18.378891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4d835-f98f-4776-a606-5cf910e3c769 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:18.379408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4d835-f98f-4776-a606-5cf910e3c769 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:18.379693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4d835-f98f-4776-a606-5cf910e3c769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:18.385722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-18e4d835-f98f-4776-a606-5cf910e3c769 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:18.507958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcc1d2-7e9d-4fae-84b9-1a9cabadb790 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:18.508458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcc1d2-7e9d-4fae-84b9-1a9cabadb790 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:18.508559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdcc1d2-7e9d-4fae-84b9-1a9cabadb790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:18.548209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4d835-f98f-4776-a606-5cf910e3c769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:18.552173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4d835-f98f-4776-a606-5cf910e3c769 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:18.556645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5cdcc1d2-7e9d-4fae-84b9-1a9cabadb790 None None] MFA Rules not processed for user `4a3f5eeb2f01498aa4c84c5ecc6a6cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:18.557895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5cdcc1d2-7e9d-4fae-84b9-1a9cabadb790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:18.585852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 989/1997] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:18.781265 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1009/1998] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:18.827322 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4d835-f98f-4776-a606-5cf910e3c769 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:18.844148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4d835-f98f-4776-a606-5cf910e3c769 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:18.848744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02f7ca-0d9f-402c-9730-5cc2a3c05c2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:18.849520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02f7ca-0d9f-402c-9730-5cc2a3c05c2f service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:18.849808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02f7ca-0d9f-402c-9730-5cc2a3c05c2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:18.850178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02f7ca-0d9f-402c-9730-5cc2a3c05c2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:18.863952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02f7ca-0d9f-402c-9730-5cc2a3c05c2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:18.866685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02f7ca-0d9f-402c-9730-5cc2a3c05c2f service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:18.905082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 990/1999] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:18.963383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcc1d2-7e9d-4fae-84b9-1a9cabadb790 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:18.964473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5cdcc1d2-7e9d-4fae-84b9-1a9cabadb790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:18.988178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdcc1d2-7e9d-4fae-84b9-1a9cabadb790 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:19.011897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:19.012823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:19.013108 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:19.013387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] PATH_INFO: `/v3/users/a17883821ba545afbda701e990d47b2c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:19.016990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a17883821ba545afbda701e990d47b2c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:19.019015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:19.047534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:19.130973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a17883821ba545afbda701e990d47b2c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:19.158242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a17883821ba545afbda701e990d47b2c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:19.170753 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a17883821ba545afbda701e990d47b2c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:19.172538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a17883821ba545afbda701e990d47b2c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:19.195269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a17883821ba545afbda701e990d47b2c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:19.213467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1010/2000] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:18 2022] DELETE /identity/v3/users/a17883821ba545afbda701e990d47b2c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:19.215203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a1c6-dc81-47c4-acf6-31e5960db357 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:19.215495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a1c6-dc81-47c4-acf6-31e5960db357 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:19.215781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a1c6-dc81-47c4-acf6-31e5960db357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:19.255980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a448a1c6-dc81-47c4-acf6-31e5960db357 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:19.257543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a448a1c6-dc81-47c4-acf6-31e5960db357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:19.349161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 991/2001] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:19.385855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:19.389086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:19.471056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5a274-ba0f-44bd-b0b6-9236cefb2102 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:19.491430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2225651-a5df-43a9-b08b-f4246589be07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:19.492537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2225651-a5df-43a9-b08b-f4246589be07 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:19.492537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2225651-a5df-43a9-b08b-f4246589be07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:19.492738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2225651-a5df-43a9-b08b-f4246589be07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:19.505709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2225651-a5df-43a9-b08b-f4246589be07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:19.507302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2225651-a5df-43a9-b08b-f4246589be07 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:19.519662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1011/2002] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:20.471291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448a1c6-dc81-47c4-acf6-31e5960db357 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:20.473681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a448a1c6-dc81-47c4-acf6-31e5960db357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:20.489915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:20.495108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448a1c6-dc81-47c4-acf6-31e5960db357 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:20.504606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:20.505719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:20.506389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:20.507313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] PATH_INFO: `/v3/projects/4c461358e18842479381971cb910c921` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:20.509095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c461358e18842479381971cb910c921)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:20.511335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:20.564455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20293cd-1279-4513-b089-a9a747e0f9dd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:20.565270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20293cd-1279-4513-b089-a9a747e0f9dd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:20.565708 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20293cd-1279-4513-b089-a9a747e0f9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:20.595512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d20293cd-1279-4513-b089-a9a747e0f9dd None None] MFA Rules not processed for user `4a3f5eeb2f01498aa4c84c5ecc6a6cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:20.596761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d20293cd-1279-4513-b089-a9a747e0f9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:20.620118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c461358e18842479381971cb910c921', 'request_id': 'req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:20.636464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c461358e18842479381971cb910c921', 'request_id': 'req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:20.645121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c461358e18842479381971cb910c921', 'request_id': 'req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:20.661383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] The token cache is being invalidate because project 4c461358e18842479381971cb910c921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:20.661999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:20.662917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:20.667989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 992/2003] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:20 2022] DELETE /identity/v3/projects/4c461358e18842479381971cb910c921 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:20.669246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:20.669885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1012/2004] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:20.674950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:20.749534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ea013-5a13-4bdc-b87f-c15f0ca32cbb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:20.756006 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:20.756895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:20.757176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:20.757454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] PATH_INFO: `/v3/users/940dd9784c7749b1960050ee8b09daac` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:20.758739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=940dd9784c7749b1960050ee8b09daac)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:20.760691 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:20.832773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '940dd9784c7749b1960050ee8b09daac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:20.833499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '940dd9784c7749b1960050ee8b09daac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:20.847197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '940dd9784c7749b1960050ee8b09daac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:20.847938 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '940dd9784c7749b1960050ee8b09daac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:20.880691 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '940dd9784c7749b1960050ee8b09daac'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:20.899267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 993/2005] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:20 2022] DELETE /identity/v3/users/940dd9784c7749b1960050ee8b09daac => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:21.247431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20293cd-1279-4513-b089-a9a747e0f9dd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:21.260933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20293cd-1279-4513-b089-a9a747e0f9dd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:21.321451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:21.348585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:21.349431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:21.349773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:21.350125 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] PATH_INFO: `/v3/projects/fcfa20f761ab4915affd8f0233a09e83` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:21.351517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcfa20f761ab4915affd8f0233a09e83)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:21.353502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:21.426374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcfa20f761ab4915affd8f0233a09e83', 'request_id': 'req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:21.434905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcfa20f761ab4915affd8f0233a09e83', 'request_id': 'req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:21.442094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcfa20f761ab4915affd8f0233a09e83', 'request_id': 'req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:21.466833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] The token cache is being invalidate because project fcfa20f761ab4915affd8f0233a09e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:21.467221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:21.467960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:21.470683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1013/2006] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:21 2022] DELETE /identity/v3/projects/fcfa20f761ab4915affd8f0233a09e83 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:21.474439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:21.475922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:21.491288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36317b7-0857-4977-a5a1-20c71463a1d5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:21.495848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:21.559357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:21.560316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:21.560673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:21.560963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] PATH_INFO: `/v3/users/a5ba731c1d3a43c797816be3e0d9b4f6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:21.562201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ba731c1d3a43c797816be3e0d9b4f6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:21.564255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:21.615568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ba731c1d3a43c797816be3e0d9b4f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:21.616294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ba731c1d3a43c797816be3e0d9b4f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:21.624945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ba731c1d3a43c797816be3e0d9b4f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:21.625590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ba731c1d3a43c797816be3e0d9b4f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:21.639664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-674a3734-0c14-4f93-bc7c-a978ce4d9c7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ba731c1d3a43c797816be3e0d9b4f6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:21.658375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 994/2007] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:21 2022] DELETE /identity/v3/users/a5ba731c1d3a43c797816be3e0d9b4f6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:21.960390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:21.961896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:21.978648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9cc6b0-bd4e-4aba-8f1f-41430d813374 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.046165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.047057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.047337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.047610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] PATH_INFO: `/v3/projects/5ea1a14ad7244d01aef20f5e3f0f49ec` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.048767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ea1a14ad7244d01aef20f5e3f0f49ec)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.050706 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.096185 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ea1a14ad7244d01aef20f5e3f0f49ec', 'request_id': 'req-7b47fb82-b134-46bf-b847-71a4460ad299'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:22.102186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ea1a14ad7244d01aef20f5e3f0f49ec', 'request_id': 'req-7b47fb82-b134-46bf-b847-71a4460ad299'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:22.108214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ea1a14ad7244d01aef20f5e3f0f49ec', 'request_id': 'req-7b47fb82-b134-46bf-b847-71a4460ad299'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:22.116495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:22.121189 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] The token cache is being invalidate because project 5ea1a14ad7244d01aef20f5e3f0f49ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:22.121674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:22.122812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7b47fb82-b134-46bf-b847-71a4460ad299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:22.125153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1014/2008] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:21 2022] DELETE /identity/v3/projects/5ea1a14ad7244d01aef20f5e3f0f49ec => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:22.131565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f557209-4293-4ec7-9f34-e068ceb46504 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.131879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f557209-4293-4ec7-9f34-e068ceb46504 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.132160 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f557209-4293-4ec7-9f34-e068ceb46504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.151300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5f557209-4293-4ec7-9f34-e068ceb46504 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:22.152495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5f557209-4293-4ec7-9f34-e068ceb46504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:22.213411 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 995/2009] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:22.222044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dd3f9-9482-42ce-95ce-cd38e8cf2a05 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.222397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dd3f9-9482-42ce-95ce-cd38e8cf2a05 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.222883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dd3f9-9482-42ce-95ce-cd38e8cf2a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.241218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-032dd3f9-9482-42ce-95ce-cd38e8cf2a05 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:22.242413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-032dd3f9-9482-42ce-95ce-cd38e8cf2a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:22.251174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1015/2010] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:22.255473 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f557209-4293-4ec7-9f34-e068ceb46504 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:22.267044 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f557209-4293-4ec7-9f34-e068ceb46504 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.271781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615accf-728c-4d69-bea2-3136efd9371a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.272443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615accf-728c-4d69-bea2-3136efd9371a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.272725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615accf-728c-4d69-bea2-3136efd9371a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.273001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615accf-728c-4d69-bea2-3136efd9371a admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.273725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615accf-728c-4d69-bea2-3136efd9371a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:22.274027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615accf-728c-4d69-bea2-3136efd9371a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.275658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615accf-728c-4d69-bea2-3136efd9371a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.290371 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 996/2011] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:05:22.291472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032dd3f9-9482-42ce-95ce-cd38e8cf2a05 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:22.301326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032dd3f9-9482-42ce-95ce-cd38e8cf2a05 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.306184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c490e2d-6dc1-45d2-8e35-f5560d6fc7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.306873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c490e2d-6dc1-45d2-8e35-f5560d6fc7fb admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.307157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c490e2d-6dc1-45d2-8e35-f5560d6fc7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.307433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c490e2d-6dc1-45d2-8e35-f5560d6fc7fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.308278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c490e2d-6dc1-45d2-8e35-f5560d6fc7fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.310174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c490e2d-6dc1-45d2-8e35-f5560d6fc7fb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.353337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1016/2012] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:22 2022] POST /identity/v3/projects => generated 409 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:22.361546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615accf-728c-4d69-bea2-3136efd9371a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:22.371496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615accf-728c-4d69-bea2-3136efd9371a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.380847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:22.427674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0a128-ac74-4a71-97a7-b3540889e705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.428332 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0a128-ac74-4a71-97a7-b3540889e705 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.428612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0a128-ac74-4a71-97a7-b3540889e705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.428888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0a128-ac74-4a71-97a7-b3540889e705 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.429571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0a128-ac74-4a71-97a7-b3540889e705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.431531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0a128-ac74-4a71-97a7-b3540889e705 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.450190 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 997/2013] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:22 2022] POST /identity/v3/users => generated 305 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:22.454255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c490e2d-6dc1-45d2-8e35-f5560d6fc7fb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:22.468144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c490e2d-6dc1-45d2-8e35-f5560d6fc7fb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.474026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a347e58-dacd-465f-b3b1-9b221c92a4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.474885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347e58-dacd-465f-b3b1-9b221c92a4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.475182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347e58-dacd-465f-b3b1-9b221c92a4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.475468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347e58-dacd-465f-b3b1-9b221c92a4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.476198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a347e58-dacd-465f-b3b1-9b221c92a4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.477912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a347e58-dacd-465f-b3b1-9b221c92a4d3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.489395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1017/2014] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:22.493387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0a128-ac74-4a71-97a7-b3540889e705 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:22.503562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0a128-ac74-4a71-97a7-b3540889e705 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.508713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e60a3-e842-4ce3-8dea-6627c881a30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.509375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e60a3-e842-4ce3-8dea-6627c881a30a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.509750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e60a3-e842-4ce3-8dea-6627c881a30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.510035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e60a3-e842-4ce3-8dea-6627c881a30a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.510897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e60a3-e842-4ce3-8dea-6627c881a30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.512500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e60a3-e842-4ce3-8dea-6627c881a30a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.522755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 998/2015] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:22.526862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a347e58-dacd-465f-b3b1-9b221c92a4d3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:22.537559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a347e58-dacd-465f-b3b1-9b221c92a4d3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.542255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f7665-f3fb-4b3d-8ce6-1999e0621752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.543183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f7665-f3fb-4b3d-8ce6-1999e0621752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.543468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f7665-f3fb-4b3d-8ce6-1999e0621752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.543746 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f7665-f3fb-4b3d-8ce6-1999e0621752 admin admin] PATH_INFO: `/v3/projects/5d309e7add2f457eb8c60cf533f0d39d/users/1553e2775e254d0e8af32a75e6d6665e/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.559600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f7665-f3fb-4b3d-8ce6-1999e0621752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d309e7add2f457eb8c60cf533f0d39d, user_id=1553e2775e254d0e8af32a75e6d6665e, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.562273 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f7665-f3fb-4b3d-8ce6-1999e0621752 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.575285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1018/2016] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:22 2022] PUT /identity/v3/projects/5d309e7add2f457eb8c60cf533f0d39d/users/1553e2775e254d0e8af32a75e6d6665e/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:22.586530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e22f1-98cf-48bd-8650-d909e064ec75 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.586985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e22f1-98cf-48bd-8650-d909e064ec75 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.587334 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e22f1-98cf-48bd-8650-d909e064ec75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.616614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e30e22f1-98cf-48bd-8650-d909e064ec75 None None] MFA Rules not processed for user `1553e2775e254d0e8af32a75e6d6665e`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:22.617978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e30e22f1-98cf-48bd-8650-d909e064ec75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:22.684698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 999/2017] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:22.689089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f7665-f3fb-4b3d-8ce6-1999e0621752 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:22.749275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f7665-f3fb-4b3d-8ce6-1999e0621752 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.753976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bc0ee-055d-479a-a4fe-81587fc04e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.754657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bc0ee-055d-479a-a4fe-81587fc04e49 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.754946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bc0ee-055d-479a-a4fe-81587fc04e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.755225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bc0ee-055d-479a-a4fe-81587fc04e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.755922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bc0ee-055d-479a-a4fe-81587fc04e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.757819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1bc0ee-055d-479a-a4fe-81587fc04e49 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.785389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1019/2018] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:22 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:22.789302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e22f1-98cf-48bd-8650-d909e064ec75 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:22.800838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e22f1-98cf-48bd-8650-d909e064ec75 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.863198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afc33e-2a15-4753-a666-795874ef0f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.863875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afc33e-2a15-4753-a666-795874ef0f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.864163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afc33e-2a15-4753-a666-795874ef0f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.864442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afc33e-2a15-4753-a666-795874ef0f34 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.865142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afc33e-2a15-4753-a666-795874ef0f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.867161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afc33e-2a15-4753-a666-795874ef0f34 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.890398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1000/2019] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:22 2022] POST /identity/v3/users => generated 311 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:22.895734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bc0ee-055d-479a-a4fe-81587fc04e49 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:22.906896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1bc0ee-055d-479a-a4fe-81587fc04e49 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.914161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560022b1-f425-4c61-ab79-f9a01e87ad29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.915438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560022b1-f425-4c61-ab79-f9a01e87ad29 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.915438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560022b1-f425-4c61-ab79-f9a01e87ad29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.915438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560022b1-f425-4c61-ab79-f9a01e87ad29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.916180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560022b1-f425-4c61-ab79-f9a01e87ad29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.917786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560022b1-f425-4c61-ab79-f9a01e87ad29 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.935062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1020/2020] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:22.936550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afc33e-2a15-4753-a666-795874ef0f34 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:22.947389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afc33e-2a15-4753-a666-795874ef0f34 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:22.952267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef971a85-f978-4da6-bf49-db8259c93bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:22.953695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971a85-f978-4da6-bf49-db8259c93bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:22.953976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971a85-f978-4da6-bf49-db8259c93bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:22.954252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef971a85-f978-4da6-bf49-db8259c93bc0 admin admin] PATH_INFO: `/v3/projects/4949621f4e6740a6b5dc86145ee61abe/users/4026013629304fec818984e8a829b9c3/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:22.969537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:22.971983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef971a85-f978-4da6-bf49-db8259c93bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4949621f4e6740a6b5dc86145ee61abe, user_id=4026013629304fec818984e8a829b9c3, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:22.974723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef971a85-f978-4da6-bf49-db8259c93bc0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:22.993347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1001/2021] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:22 2022] PUT /identity/v3/projects/4949621f4e6740a6b5dc86145ee61abe/users/4026013629304fec818984e8a829b9c3/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:22.997781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560022b1-f425-4c61-ab79-f9a01e87ad29 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:23.008677 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560022b1-f425-4c61-ab79-f9a01e87ad29 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:23.073524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb4ae0-2c24-43ee-9b8e-e71f596edf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:23.074370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb4ae0-2c24-43ee-9b8e-e71f596edf7b admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:23.074784 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb4ae0-2c24-43ee-9b8e-e71f596edf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:23.075158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb4ae0-2c24-43ee-9b8e-e71f596edf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:23.076764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb4ae0-2c24-43ee-9b8e-e71f596edf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:23.079032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb4ae0-2c24-43ee-9b8e-e71f596edf7b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:23.091913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1021/2022] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:23.096519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef971a85-f978-4da6-bf49-db8259c93bc0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:23.108753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef971a85-f978-4da6-bf49-db8259c93bc0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:23.115135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3841c2-8eaf-4080-b056-c68b348090cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:23.116135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3841c2-8eaf-4080-b056-c68b348090cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:23.116541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3841c2-8eaf-4080-b056-c68b348090cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:23.116873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3841c2-8eaf-4080-b056-c68b348090cc admin admin] PATH_INFO: `/v3/projects/4949621f4e6740a6b5dc86145ee61abe/users/4026013629304fec818984e8a829b9c3/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:23.119677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3841c2-8eaf-4080-b056-c68b348090cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4949621f4e6740a6b5dc86145ee61abe, user_id=4026013629304fec818984e8a829b9c3, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:23.123617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3841c2-8eaf-4080-b056-c68b348090cc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:23.142218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1002/2023] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:23 2022] PUT /identity/v3/projects/4949621f4e6740a6b5dc86145ee61abe/users/4026013629304fec818984e8a829b9c3/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:23.153451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afe23b-bc27-4327-b8b1-f898d339562f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:23.153745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afe23b-bc27-4327-b8b1-f898d339562f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:23.154027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afe23b-bc27-4327-b8b1-f898d339562f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:23.167578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:23.192394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-26afe23b-bc27-4327-b8b1-f898d339562f None None] MFA Rules not processed for user `4026013629304fec818984e8a829b9c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:23.193643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-26afe23b-bc27-4327-b8b1-f898d339562f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:23.263446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1022/2024] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:23.275723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3841c2-8eaf-4080-b056-c68b348090cc admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:23.277547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6e3841c2-8eaf-4080-b056-c68b348090cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:23.337772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3841c2-8eaf-4080-b056-c68b348090cc admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:23.341580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369d4e2-2d7e-4db6-a420-a8bdf159e2ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:23.342226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369d4e2-2d7e-4db6-a420-a8bdf159e2ff service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:23.342549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369d4e2-2d7e-4db6-a420-a8bdf159e2ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:23.342834 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369d4e2-2d7e-4db6-a420-a8bdf159e2ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:23.351826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369d4e2-2d7e-4db6-a420-a8bdf159e2ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:23.353282 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369d4e2-2d7e-4db6-a420-a8bdf159e2ff service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:23.363043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1003/2025] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:23 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:23.558425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afe23b-bc27-4327-b8b1-f898d339562f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:23.570233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afe23b-bc27-4327-b8b1-f898d339562f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:23.574182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22eadab-cd5b-4478-8217-ef1cbec48417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:23.575377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22eadab-cd5b-4478-8217-ef1cbec48417 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:23.575377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22eadab-cd5b-4478-8217-ef1cbec48417 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:23.575674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22eadab-cd5b-4478-8217-ef1cbec48417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:23.646672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22eadab-cd5b-4478-8217-ef1cbec48417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:23.648438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22eadab-cd5b-4478-8217-ef1cbec48417 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:23.658912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1023/2026] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:24.904960 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369d4e2-2d7e-4db6-a420-a8bdf159e2ff service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:24.907221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6369d4e2-2d7e-4db6-a420-a8bdf159e2ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:24.978927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369d4e2-2d7e-4db6-a420-a8bdf159e2ff service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:24.985892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:24.986876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:24.987241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:24.987451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] PATH_INFO: `/v3/users/4243daae2d1243b59e179b3e2c0ca1d7` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:24.988718 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4243daae2d1243b59e179b3e2c0ca1d7)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:24.991071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:25.045059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4243daae2d1243b59e179b3e2c0ca1d7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.045717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4243daae2d1243b59e179b3e2c0ca1d7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.055022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4243daae2d1243b59e179b3e2c0ca1d7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.055637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4243daae2d1243b59e179b3e2c0ca1d7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.068098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4243daae2d1243b59e179b3e2c0ca1d7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.080121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:25.081737 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1004/2027] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:24 2022] DELETE /identity/v3/users/4243daae2d1243b59e179b3e2c0ca1d7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:25.086541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d178b9-b2c2-43b3-a7d5-f32d5d40aa3f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:25.087081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d178b9-b2c2-43b3-a7d5-f32d5d40aa3f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:25.087181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d178b9-b2c2-43b3-a7d5-f32d5d40aa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:25.125524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-24d178b9-b2c2-43b3-a7d5-f32d5d40aa3f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:25.127202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-24d178b9-b2c2-43b3-a7d5-f32d5d40aa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:25.211797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1024/2028] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:25.226947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:25.229102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:25.298233 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8117fb9-d3b8-405a-8861-72287dab7709 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:25.301975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c377ae1-5fcb-45f9-bda7-063865b0e8ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:25.302660 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c377ae1-5fcb-45f9-bda7-063865b0e8ea service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:25.302949 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c377ae1-5fcb-45f9-bda7-063865b0e8ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:25.303229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c377ae1-5fcb-45f9-bda7-063865b0e8ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:25.313108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c377ae1-5fcb-45f9-bda7-063865b0e8ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:25.314654 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c377ae1-5fcb-45f9-bda7-063865b0e8ea service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:25.324964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1005/2029] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:25.598812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d178b9-b2c2-43b3-a7d5-f32d5d40aa3f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:25.600250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-24d178b9-b2c2-43b3-a7d5-f32d5d40aa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:25.612298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d178b9-b2c2-43b3-a7d5-f32d5d40aa3f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:25.617684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:25.618784 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:25.619011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:25.619276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] PATH_INFO: `/v3/projects/971dd1a0b88e4fa9ba667db8983a127e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:25.624270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=971dd1a0b88e4fa9ba667db8983a127e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:25.624270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:25.654907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:25.678536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '971dd1a0b88e4fa9ba667db8983a127e', 'request_id': 'req-09b325d3-7e14-42fb-9b3b-4925b7e01957'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.684860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '971dd1a0b88e4fa9ba667db8983a127e', 'request_id': 'req-09b325d3-7e14-42fb-9b3b-4925b7e01957'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.691872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '971dd1a0b88e4fa9ba667db8983a127e', 'request_id': 'req-09b325d3-7e14-42fb-9b3b-4925b7e01957'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.707302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] The token cache is being invalidate because project 971dd1a0b88e4fa9ba667db8983a127e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:25.707598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.708265 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.709818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1025/2030] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:25 2022] DELETE /identity/v3/projects/971dd1a0b88e4fa9ba667db8983a127e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:25.713626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c377ae1-5fcb-45f9-bda7-063865b0e8ea service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:25.725998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c377ae1-5fcb-45f9-bda7-063865b0e8ea service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:25.784478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:25.785380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:25.785664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:25.785944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] PATH_INFO: `/v3/users/807255e520214fa9a2e4c05d6400dfa3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:25.787154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=807255e520214fa9a2e4c05d6400dfa3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:25.789119 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:25.849287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807255e520214fa9a2e4c05d6400dfa3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.850092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807255e520214fa9a2e4c05d6400dfa3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.862530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807255e520214fa9a2e4c05d6400dfa3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.863315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807255e520214fa9a2e4c05d6400dfa3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.882323 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f36bde7-9d46-4e2f-8e82-686ef37b7229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807255e520214fa9a2e4c05d6400dfa3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:25.918571 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1006/2031] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:25 2022] DELETE /identity/v3/users/807255e520214fa9a2e4c05d6400dfa3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:25.975981 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:25.977446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:25.991312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b325d3-7e14-42fb-9b3b-4925b7e01957 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:26.055696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:26.056577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:26.056857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:26.057130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] PATH_INFO: `/v3/users/f2222e0928d3481e97646167edf90f8b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:26.058546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2222e0928d3481e97646167edf90f8b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:26.060520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:26.113036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2222e0928d3481e97646167edf90f8b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.125962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2222e0928d3481e97646167edf90f8b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.133479 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2222e0928d3481e97646167edf90f8b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.134097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2222e0928d3481e97646167edf90f8b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.141565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2222e0928d3481e97646167edf90f8b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.147610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1026/2032] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:25 2022] DELETE /identity/v3/users/f2222e0928d3481e97646167edf90f8b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:26.152069 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6437d-b36d-40ca-8e86-e714ca4b7405 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:26.152359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6437d-b36d-40ca-8e86-e714ca4b7405 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:26.152641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6437d-b36d-40ca-8e86-e714ca4b7405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:26.169372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-23f6437d-b36d-40ca-8e86-e714ca4b7405 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:26.170619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-23f6437d-b36d-40ca-8e86-e714ca4b7405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:26.182950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:26.222271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:26.224229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:26.263278 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1007/2033] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:26.265468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6437d-b36d-40ca-8e86-e714ca4b7405 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:26.266654 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-23f6437d-b36d-40ca-8e86-e714ca4b7405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:26.290622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:26.323450 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698adf1-dd31-4c38-87c9-951890ef2ac9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:26.328255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:26.329188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:26.329543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:26.329891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] PATH_INFO: `/v3/projects/ab6ad362411042b0b592d85a1d684aee` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:26.330002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6437d-b36d-40ca-8e86-e714ca4b7405 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:26.331227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6ad362411042b0b592d85a1d684aee)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:26.333136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:26.333626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26480f2-1901-4085-8e37-01ad91a5198d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:26.334264 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26480f2-1901-4085-8e37-01ad91a5198d service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:26.334591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26480f2-1901-4085-8e37-01ad91a5198d service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:26.334867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26480f2-1901-4085-8e37-01ad91a5198d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:26.344460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26480f2-1901-4085-8e37-01ad91a5198d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:26.345821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26480f2-1901-4085-8e37-01ad91a5198d service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:26.355388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1008/2034] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:05:26.382863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6ad362411042b0b592d85a1d684aee', 'request_id': 'req-2003f7e5-7399-4952-bfb2-0a14d3075d25'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.389405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6ad362411042b0b592d85a1d684aee', 'request_id': 'req-2003f7e5-7399-4952-bfb2-0a14d3075d25'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.395571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6ad362411042b0b592d85a1d684aee', 'request_id': 'req-2003f7e5-7399-4952-bfb2-0a14d3075d25'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.409903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] The token cache is being invalidate because project ab6ad362411042b0b592d85a1d684aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:26.410221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.410914 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.412762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1027/2035] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:26 2022] DELETE /identity/v3/projects/ab6ad362411042b0b592d85a1d684aee => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:26.417207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26480f2-1901-4085-8e37-01ad91a5198d service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:26.431029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26480f2-1901-4085-8e37-01ad91a5198d service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:26.493162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:26.494032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:26.494314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:26.494634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] PATH_INFO: `/v3/users/5a833289bea84f12aed5e0c42a610535` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:26.495846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a833289bea84f12aed5e0c42a610535)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:26.497798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:26.546479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a833289bea84f12aed5e0c42a610535'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.547688 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a833289bea84f12aed5e0c42a610535'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.559816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a833289bea84f12aed5e0c42a610535'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.560687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a833289bea84f12aed5e0c42a610535'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.570414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:26.576682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a833289bea84f12aed5e0c42a610535'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.592679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1009/2036] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:26 2022] DELETE /identity/v3/users/5a833289bea84f12aed5e0c42a610535 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:26.660444 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:26.662233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:26.676864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003f7e5-7399-4952-bfb2-0a14d3075d25 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:26.738845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:26.740012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:26.740306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:26.740586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] PATH_INFO: `/v3/projects/19d4ead6e092498daaf1d4ddb55a7653` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:26.741791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d4ead6e092498daaf1d4ddb55a7653)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:26.743784 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:26.794723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d4ead6e092498daaf1d4ddb55a7653', 'request_id': 'req-b50817a3-1b66-4e03-a262-8bce277dfab7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.801490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d4ead6e092498daaf1d4ddb55a7653', 'request_id': 'req-b50817a3-1b66-4e03-a262-8bce277dfab7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.808054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d4ead6e092498daaf1d4ddb55a7653', 'request_id': 'req-b50817a3-1b66-4e03-a262-8bce277dfab7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.809990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:26.827780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] The token cache is being invalidate because project 19d4ead6e092498daaf1d4ddb55a7653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:26.828308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.830363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.832087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1028/2037] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:26 2022] DELETE /identity/v3/projects/19d4ead6e092498daaf1d4ddb55a7653 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:26.836463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:26.838148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:26.854229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2af555-dc46-44d6-a874-f24609e357e5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:26.881578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:26.883231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:26.924090 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:26.925342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:26.925699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:26.925986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] PATH_INFO: `/v3/users/35c3d92af9c84f30a072877e0f37b356` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:26.927373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c3d92af9c84f30a072877e0f37b356)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:26.929463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:26.965097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50817a3-1b66-4e03-a262-8bce277dfab7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:26.970597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:26.971467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:26.971751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:26.972028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] PATH_INFO: `/v3/projects/0779edfd81be48a690675589940b1558` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:26.973213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0779edfd81be48a690675589940b1558)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:26.975177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:26.982783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c3d92af9c84f30a072877e0f37b356'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.983865 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c3d92af9c84f30a072877e0f37b356'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.994142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c3d92af9c84f30a072877e0f37b356'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:26.995134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c3d92af9c84f30a072877e0f37b356'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.009948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c3d92af9c84f30a072877e0f37b356'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.023671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1010/2038] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:26 2022] DELETE /identity/v3/users/35c3d92af9c84f30a072877e0f37b356 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:27.025997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0779edfd81be48a690675589940b1558', 'request_id': 'req-e1863d4d-6a40-4a15-b865-72ef70cdd332'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.033447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0779edfd81be48a690675589940b1558', 'request_id': 'req-e1863d4d-6a40-4a15-b865-72ef70cdd332'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.040693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0779edfd81be48a690675589940b1558', 'request_id': 'req-e1863d4d-6a40-4a15-b865-72ef70cdd332'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.055964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] The token cache is being invalidate because project 0779edfd81be48a690675589940b1558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:27.056270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.057266 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1863d4d-6a40-4a15-b865-72ef70cdd332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.066540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1029/2039] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:26 2022] DELETE /identity/v3/projects/0779edfd81be48a690675589940b1558 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:27.353072 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:27.354677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:27.422156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16def64d-692f-4f7e-8a94-56626c0bdbfa admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:27.426694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:27.427567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:27.427849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:27.428125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] PATH_INFO: `/v3/projects/5fa25ced23894e05b224a0acd5bc144c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:27.429292 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa25ced23894e05b224a0acd5bc144c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:27.431243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:27.432699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:27.479832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa25ced23894e05b224a0acd5bc144c', 'request_id': 'req-f85e324e-f8b2-4820-922d-c74dc222e9f4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.485261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa25ced23894e05b224a0acd5bc144c', 'request_id': 'req-f85e324e-f8b2-4820-922d-c74dc222e9f4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.491228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa25ced23894e05b224a0acd5bc144c', 'request_id': 'req-f85e324e-f8b2-4820-922d-c74dc222e9f4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.504593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] The token cache is being invalidate because project 5fa25ced23894e05b224a0acd5bc144c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:27.504888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.505643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f85e324e-f8b2-4820-922d-c74dc222e9f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:27.507165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1011/2040] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:27 2022] DELETE /identity/v3/projects/5fa25ced23894e05b224a0acd5bc144c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:27.512052 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f897-574f-4687-9953-47f8d5fa7e69 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:27.512556 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f897-574f-4687-9953-47f8d5fa7e69 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:27.512681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f897-574f-4687-9953-47f8d5fa7e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:27.532044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-af74f897-574f-4687-9953-47f8d5fa7e69 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:27.533371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-af74f897-574f-4687-9953-47f8d5fa7e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:27.597576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1030/2041] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:27.605061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcca499-6527-405d-8f2c-83114f2d3cb1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:27.605384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcca499-6527-405d-8f2c-83114f2d3cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:27.605670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcca499-6527-405d-8f2c-83114f2d3cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:27.624130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ffcca499-6527-405d-8f2c-83114f2d3cb1 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:27.625357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ffcca499-6527-405d-8f2c-83114f2d3cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:27.633292 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1012/2042] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:27.637526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74f897-574f-4687-9953-47f8d5fa7e69 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:27.649583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74f897-574f-4687-9953-47f8d5fa7e69 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:27.654241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd9451-6f4c-4a88-8bfe-7f102402cba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:27.654943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd9451-6f4c-4a88-8bfe-7f102402cba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:27.655234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd9451-6f4c-4a88-8bfe-7f102402cba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:27.655519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd9451-6f4c-4a88-8bfe-7f102402cba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:27.656794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd9451-6f4c-4a88-8bfe-7f102402cba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:27.657110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd9451-6f4c-4a88-8bfe-7f102402cba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:27.658741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd9451-6f4c-4a88-8bfe-7f102402cba2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:27.670226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1031/2043] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:05:27.674570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcca499-6527-405d-8f2c-83114f2d3cb1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:27.683702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcca499-6527-405d-8f2c-83114f2d3cb1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:27.688793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425458dc-abd0-4b64-abf3-00c21670832b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:27.689435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425458dc-abd0-4b64-abf3-00c21670832b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:27.689721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425458dc-abd0-4b64-abf3-00c21670832b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:27.690000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425458dc-abd0-4b64-abf3-00c21670832b admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:27.690711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425458dc-abd0-4b64-abf3-00c21670832b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:27.692622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425458dc-abd0-4b64-abf3-00c21670832b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:27.718380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1013/2044] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:27 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:27.722381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd9451-6f4c-4a88-8bfe-7f102402cba2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:27.732332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd9451-6f4c-4a88-8bfe-7f102402cba2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:27.737069 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:27.789671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e4bbe-efa1-4136-bace-db97d5187d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:27.790334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e4bbe-efa1-4136-bace-db97d5187d31 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:27.790651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e4bbe-efa1-4136-bace-db97d5187d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:27.790927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e4bbe-efa1-4136-bace-db97d5187d31 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:27.791601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e4bbe-efa1-4136-bace-db97d5187d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:27.793496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e4bbe-efa1-4136-bace-db97d5187d31 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:27.814035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1032/2045] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:27 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:27.818108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425458dc-abd0-4b64-abf3-00c21670832b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:27.829013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425458dc-abd0-4b64-abf3-00c21670832b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:27.833776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3457e1-4d7e-491b-856c-7a6798e09567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:27.834438 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3457e1-4d7e-491b-856c-7a6798e09567 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:27.834756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3457e1-4d7e-491b-856c-7a6798e09567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:27.835035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3457e1-4d7e-491b-856c-7a6798e09567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:27.835762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3457e1-4d7e-491b-856c-7a6798e09567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:27.837426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3457e1-4d7e-491b-856c-7a6798e09567 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:27.848247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1014/2046] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:27.852103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e4bbe-efa1-4136-bace-db97d5187d31 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:27.862537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e4bbe-efa1-4136-bace-db97d5187d31 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:27.868098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cd0e4-f651-44b7-995f-a2b31eed3351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:27.868779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cd0e4-f651-44b7-995f-a2b31eed3351 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:27.869296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cd0e4-f651-44b7-995f-a2b31eed3351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:27.869797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cd0e4-f651-44b7-995f-a2b31eed3351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:27.870931 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cd0e4-f651-44b7-995f-a2b31eed3351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:27.872523 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cd0e4-f651-44b7-995f-a2b31eed3351 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:27.884765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1033/2047] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:05:27.889746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3457e1-4d7e-491b-856c-7a6798e09567 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:27.900339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3457e1-4d7e-491b-856c-7a6798e09567 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:27.905977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec26fb-b277-4596-912d-e9fe1a31c735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:27.907422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec26fb-b277-4596-912d-e9fe1a31c735 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:27.907705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec26fb-b277-4596-912d-e9fe1a31c735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:27.908002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec26fb-b277-4596-912d-e9fe1a31c735 admin admin] PATH_INFO: `/v3/projects/2edda53a2da04945bf3d7b7ea60de3a0/users/24c21881b22e40878a69689b63155484/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:27.922843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec26fb-b277-4596-912d-e9fe1a31c735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2edda53a2da04945bf3d7b7ea60de3a0, user_id=24c21881b22e40878a69689b63155484, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:27.925572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dec26fb-b277-4596-912d-e9fe1a31c735 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:27.938393 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1015/2048] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:27 2022] PUT /identity/v3/projects/2edda53a2da04945bf3d7b7ea60de3a0/users/24c21881b22e40878a69689b63155484/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:27.948676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00549a4a-0708-4614-a779-eb51c93a3996 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:27.949002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00549a4a-0708-4614-a779-eb51c93a3996 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:27.949283 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00549a4a-0708-4614-a779-eb51c93a3996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:27.975581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-00549a4a-0708-4614-a779-eb51c93a3996 None None] MFA Rules not processed for user `24c21881b22e40878a69689b63155484`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:27.976777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-00549a4a-0708-4614-a779-eb51c93a3996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:28.038812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1034/2049] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:28.043228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec26fb-b277-4596-912d-e9fe1a31c735 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:28.106283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec26fb-b277-4596-912d-e9fe1a31c735 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:28.111240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ceab-26be-4772-8728-f085a6cad681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:28.111897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ceab-26be-4772-8728-f085a6cad681 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:28.112182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ceab-26be-4772-8728-f085a6cad681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:28.112462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ceab-26be-4772-8728-f085a6cad681 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:28.113152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111ceab-26be-4772-8728-f085a6cad681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:28.115112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111ceab-26be-4772-8728-f085a6cad681 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:28.141837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1016/2050] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:28 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:28.145779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00549a4a-0708-4614-a779-eb51c93a3996 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:28.157401 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00549a4a-0708-4614-a779-eb51c93a3996 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:28.215309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b109fb6-8fcb-47f9-9475-908a311f63ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:28.215945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b109fb6-8fcb-47f9-9475-908a311f63ec admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:28.216231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b109fb6-8fcb-47f9-9475-908a311f63ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:28.216512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b109fb6-8fcb-47f9-9475-908a311f63ec admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:28.217213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b109fb6-8fcb-47f9-9475-908a311f63ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:28.219855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b109fb6-8fcb-47f9-9475-908a311f63ec admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:28.242118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1035/2051] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:28 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:28.247970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ceab-26be-4772-8728-f085a6cad681 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:28.266337 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ceab-26be-4772-8728-f085a6cad681 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:28.272797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d5b48-82a4-4d60-a30b-08bc2c5d5a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:28.274480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d5b48-82a4-4d60-a30b-08bc2c5d5a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:28.274769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d5b48-82a4-4d60-a30b-08bc2c5d5a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:28.275049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d5b48-82a4-4d60-a30b-08bc2c5d5a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:28.275782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d5b48-82a4-4d60-a30b-08bc2c5d5a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:28.277410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d5b48-82a4-4d60-a30b-08bc2c5d5a84 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:28.283829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:28.295823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1017/2052] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:28.299513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b109fb6-8fcb-47f9-9475-908a311f63ec admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:28.311417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b109fb6-8fcb-47f9-9475-908a311f63ec admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:28.320552 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286392e7-22b5-429b-83ca-8770b183c8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:28.322462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286392e7-22b5-429b-83ca-8770b183c8e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:28.322830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286392e7-22b5-429b-83ca-8770b183c8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:28.323124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286392e7-22b5-429b-83ca-8770b183c8e1 admin admin] PATH_INFO: `/v3/projects/df273ffbc54c46f0ab8d9daffbfa0a4f/users/df840ff1e4e348838074ee6c39e67afe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:28.327820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:28.347458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286392e7-22b5-429b-83ca-8770b183c8e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df273ffbc54c46f0ab8d9daffbfa0a4f, user_id=df840ff1e4e348838074ee6c39e67afe, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:28.350248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286392e7-22b5-429b-83ca-8770b183c8e1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:28.365327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1036/2053] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:28 2022] PUT /identity/v3/projects/df273ffbc54c46f0ab8d9daffbfa0a4f/users/df840ff1e4e348838074ee6c39e67afe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:28.369368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d5b48-82a4-4d60-a30b-08bc2c5d5a84 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:28.380594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d5b48-82a4-4d60-a30b-08bc2c5d5a84 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:28.441335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbbaeb7-fb8e-485b-bfae-1ed536470b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:28.442001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbaeb7-fb8e-485b-bfae-1ed536470b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:28.442285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbaeb7-fb8e-485b-bfae-1ed536470b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:28.442588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbbaeb7-fb8e-485b-bfae-1ed536470b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:28.443308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbbaeb7-fb8e-485b-bfae-1ed536470b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:28.444893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbbaeb7-fb8e-485b-bfae-1ed536470b29 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:28.455352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1018/2054] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:28.459539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286392e7-22b5-429b-83ca-8770b183c8e1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:28.469782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286392e7-22b5-429b-83ca-8770b183c8e1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:28.474339 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf76c51-e5c9-4137-bb70-f6d96a060db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:28.475246 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf76c51-e5c9-4137-bb70-f6d96a060db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:28.475534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf76c51-e5c9-4137-bb70-f6d96a060db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:28.475813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf76c51-e5c9-4137-bb70-f6d96a060db5 admin admin] PATH_INFO: `/v3/projects/df273ffbc54c46f0ab8d9daffbfa0a4f/users/df840ff1e4e348838074ee6c39e67afe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:28.477780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf76c51-e5c9-4137-bb70-f6d96a060db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df273ffbc54c46f0ab8d9daffbfa0a4f, user_id=df840ff1e4e348838074ee6c39e67afe, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:28.480498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf76c51-e5c9-4137-bb70-f6d96a060db5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:28.490494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1037/2055] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:28 2022] PUT /identity/v3/projects/df273ffbc54c46f0ab8d9daffbfa0a4f/users/df840ff1e4e348838074ee6c39e67afe/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:28.501937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d111443-2897-4db5-996d-b9a268dba231 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:28.502228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d111443-2897-4db5-996d-b9a268dba231 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:28.502546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d111443-2897-4db5-996d-b9a268dba231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:28.528751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5d111443-2897-4db5-996d-b9a268dba231 None None] MFA Rules not processed for user `df840ff1e4e348838074ee6c39e67afe`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:28.529949 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5d111443-2897-4db5-996d-b9a268dba231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:28.597965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1019/2056] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:28.615484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf76c51-e5c9-4137-bb70-f6d96a060db5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:28.617359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eaf76c51-e5c9-4137-bb70-f6d96a060db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:28.718198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf76c51-e5c9-4137-bb70-f6d96a060db5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:28.725034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a25dc-282e-4dd7-a031-29ff5da5513e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:28.726091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a25dc-282e-4dd7-a031-29ff5da5513e service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:28.726776 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a25dc-282e-4dd7-a031-29ff5da5513e service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:28.727245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a25dc-282e-4dd7-a031-29ff5da5513e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:28.743357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a25dc-282e-4dd7-a031-29ff5da5513e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:28.746194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a25dc-282e-4dd7-a031-29ff5da5513e service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:28.765234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1038/2057] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:05:28.844718 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d111443-2897-4db5-996d-b9a268dba231 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:28.854107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d111443-2897-4db5-996d-b9a268dba231 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:28.857612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e66e9-ecda-4cab-95af-adb46de2320d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:28.858240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e66e9-ecda-4cab-95af-adb46de2320d service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:28.858550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e66e9-ecda-4cab-95af-adb46de2320d service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:28.858828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e66e9-ecda-4cab-95af-adb46de2320d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:28.913948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e66e9-ecda-4cab-95af-adb46de2320d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:28.915440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e66e9-ecda-4cab-95af-adb46de2320d service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:28.924851 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1020/2058] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:32.206091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a25dc-282e-4dd7-a031-29ff5da5513e service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:32.207736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6b1a25dc-282e-4dd7-a031-29ff5da5513e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:32.268566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e66e9-ecda-4cab-95af-adb46de2320d service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:32.270162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-013e66e9-ecda-4cab-95af-adb46de2320d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:32.279990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a25dc-282e-4dd7-a031-29ff5da5513e service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:32.284477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e66e9-ecda-4cab-95af-adb46de2320d service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:32.285208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:32.286116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:32.286405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:32.286712 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] PATH_INFO: `/v3/users/d1e6e50416af41d1af7d18cc5fb90253` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:32.287893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1e6e50416af41d1af7d18cc5fb90253)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:32.288950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:32.289828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:32.289935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:32.290120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:32.290396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] PATH_INFO: `/v3/users/1553e2775e254d0e8af32a75e6d6665e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:32.291649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1553e2775e254d0e8af32a75e6d6665e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:32.293604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:32.308677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:32.352600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1553e2775e254d0e8af32a75e6d6665e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.353249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1553e2775e254d0e8af32a75e6d6665e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.353971 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1e6e50416af41d1af7d18cc5fb90253'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.361911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1553e2775e254d0e8af32a75e6d6665e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.362591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1553e2775e254d0e8af32a75e6d6665e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.372733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1e6e50416af41d1af7d18cc5fb90253'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.375848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8af6cbaa-0451-4201-9bb6-a21b3427222c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1553e2775e254d0e8af32a75e6d6665e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.382700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1e6e50416af41d1af7d18cc5fb90253'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.383732 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1e6e50416af41d1af7d18cc5fb90253'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.388270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:32.389562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1021/2059] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:32 2022] DELETE /identity/v3/users/1553e2775e254d0e8af32a75e6d6665e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:32.394611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6048ef9-49f2-435b-aae2-73a83e301922 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:32.395034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6048ef9-49f2-435b-aae2-73a83e301922 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:32.395317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6048ef9-49f2-435b-aae2-73a83e301922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:32.396553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1e6e50416af41d1af7d18cc5fb90253'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.404738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1039/2060] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:32 2022] DELETE /identity/v3/users/d1e6e50416af41d1af7d18cc5fb90253 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:32.417822 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c6048ef9-49f2-435b-aae2-73a83e301922 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:32.419321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c6048ef9-49f2-435b-aae2-73a83e301922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:32.493511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1022/2061] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:32.502964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:32.504593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:32.563822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a611dee-43d5-4d84-80a0-555b286a8750 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:32.567396 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e762fdd-4a55-4d86-b6a0-a516715f21ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:32.568331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e762fdd-4a55-4d86-b6a0-a516715f21ec service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:32.568615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e762fdd-4a55-4d86-b6a0-a516715f21ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:32.568890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e762fdd-4a55-4d86-b6a0-a516715f21ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:32.579622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e762fdd-4a55-4d86-b6a0-a516715f21ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:32.581336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e762fdd-4a55-4d86-b6a0-a516715f21ec service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:32.587364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6048ef9-49f2-435b-aae2-73a83e301922 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:32.589016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c6048ef9-49f2-435b-aae2-73a83e301922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:32.592417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1040/2062] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:05:32.606128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6048ef9-49f2-435b-aae2-73a83e301922 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:32.612817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:32.613692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:32.613976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:32.614297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] PATH_INFO: `/v3/projects/dab98c8a7d9e43249b01602eb3b63bb8` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:32.615466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dab98c8a7d9e43249b01602eb3b63bb8)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:32.617746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:32.670794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dab98c8a7d9e43249b01602eb3b63bb8', 'request_id': 'req-cf5fa191-9d66-47f8-ad85-212465104166'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.676595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dab98c8a7d9e43249b01602eb3b63bb8', 'request_id': 'req-cf5fa191-9d66-47f8-ad85-212465104166'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.682857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dab98c8a7d9e43249b01602eb3b63bb8', 'request_id': 'req-cf5fa191-9d66-47f8-ad85-212465104166'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.696284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] The token cache is being invalidate because project dab98c8a7d9e43249b01602eb3b63bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:32.696573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.697236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-cf5fa191-9d66-47f8-ad85-212465104166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.698836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1023/2063] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:32 2022] DELETE /identity/v3/projects/dab98c8a7d9e43249b01602eb3b63bb8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:32.704522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d04b1-3035-49c9-96c1-ae45114ffaf7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:32.704813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d04b1-3035-49c9-96c1-ae45114ffaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:32.705091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d04b1-3035-49c9-96c1-ae45114ffaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:32.723692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5a5d04b1-3035-49c9-96c1-ae45114ffaf7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:32.724895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5a5d04b1-3035-49c9-96c1-ae45114ffaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:32.788155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1041/2064] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:32.796011 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53516a3-f312-47b3-95cb-6605a961bc5a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:32.796367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53516a3-f312-47b3-95cb-6605a961bc5a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:32.796746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53516a3-f312-47b3-95cb-6605a961bc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:32.821520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f53516a3-f312-47b3-95cb-6605a961bc5a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:32.822653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f53516a3-f312-47b3-95cb-6605a961bc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:32.832484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1024/2065] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:32.836564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d04b1-3035-49c9-96c1-ae45114ffaf7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:32.847441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d04b1-3035-49c9-96c1-ae45114ffaf7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:32.852130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc3216-0858-4a96-8d6f-fafb34907687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:32.852790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc3216-0858-4a96-8d6f-fafb34907687 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:32.853075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc3216-0858-4a96-8d6f-fafb34907687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:32.862611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc3216-0858-4a96-8d6f-fafb34907687 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:32.863827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc3216-0858-4a96-8d6f-fafb34907687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:32.864182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc3216-0858-4a96-8d6f-fafb34907687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:32.865886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc3216-0858-4a96-8d6f-fafb34907687 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:32.879635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1042/2066] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:05:32.884259 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53516a3-f312-47b3-95cb-6605a961bc5a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:32.884392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc3216-0858-4a96-8d6f-fafb34907687 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:32.886750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-21bc3216-0858-4a96-8d6f-fafb34907687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:32.894514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53516a3-f312-47b3-95cb-6605a961bc5a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:32.901035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc3216-0858-4a96-8d6f-fafb34907687 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:32.901211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2630f70-bee5-43f4-81d3-a8f99d571525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:32.901798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2630f70-bee5-43f4-81d3-a8f99d571525 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:32.902098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2630f70-bee5-43f4-81d3-a8f99d571525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:32.902363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2630f70-bee5-43f4-81d3-a8f99d571525 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:32.903222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2630f70-bee5-43f4-81d3-a8f99d571525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:32.905198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2630f70-bee5-43f4-81d3-a8f99d571525 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:32.907396 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:32.908257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:32.908541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:32.908816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] PATH_INFO: `/v3/projects/5d309e7add2f457eb8c60cf533f0d39d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:32.910171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d309e7add2f457eb8c60cf533f0d39d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:32.911400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:32.912221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:32.951236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1025/2067] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:32 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:32.956059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2630f70-bee5-43f4-81d3-a8f99d571525 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:32.977959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d309e7add2f457eb8c60cf533f0d39d', 'request_id': 'req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.985371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d309e7add2f457eb8c60cf533f0d39d', 'request_id': 'req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:32.992761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d309e7add2f457eb8c60cf533f0d39d', 'request_id': 'req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.010236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] The token cache is being invalidate because project 5d309e7add2f457eb8c60cf533f0d39d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:33.010642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.011348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.012985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1043/2068] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:32 2022] DELETE /identity/v3/projects/5d309e7add2f457eb8c60cf533f0d39d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:33.017241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:33.018646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:33.024477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:33.028123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2630f70-bee5-43f4-81d3-a8f99d571525 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:33.086952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50ed78-0245-498b-b3d0-b93b9ce2dde7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:33.087168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2185d68-46d8-4c83-bb72-ee7c6654e250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:33.087711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2185d68-46d8-4c83-bb72-ee7c6654e250 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.088072 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2185d68-46d8-4c83-bb72-ee7c6654e250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.088423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2185d68-46d8-4c83-bb72-ee7c6654e250 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.089189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2185d68-46d8-4c83-bb72-ee7c6654e250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:33.091223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2185d68-46d8-4c83-bb72-ee7c6654e250 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:33.091622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:33.092484 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.092766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.093042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] PATH_INFO: `/v3/users/4026013629304fec818984e8a829b9c3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.094297 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=4026013629304fec818984e8a829b9c3)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:33.096245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:33.112531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1026/2069] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:32 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:33.116533 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2185d68-46d8-4c83-bb72-ee7c6654e250 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:33.118151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a2185d68-46d8-4c83-bb72-ee7c6654e250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:33.131766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2185d68-46d8-4c83-bb72-ee7c6654e250 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:33.136723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea365b-fca1-4993-ba73-cfee7d56f9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:33.137422 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea365b-fca1-4993-ba73-cfee7d56f9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.137711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea365b-fca1-4993-ba73-cfee7d56f9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.137982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea365b-fca1-4993-ba73-cfee7d56f9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.138758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea365b-fca1-4993-ba73-cfee7d56f9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:33.140947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea365b-fca1-4993-ba73-cfee7d56f9b3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:33.144059 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4026013629304fec818984e8a829b9c3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.150828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1027/2070] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:33.154670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea365b-fca1-4993-ba73-cfee7d56f9b3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:33.159462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4026013629304fec818984e8a829b9c3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.164189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea365b-fca1-4993-ba73-cfee7d56f9b3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:33.166849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4026013629304fec818984e8a829b9c3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.167549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4026013629304fec818984e8a829b9c3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.169026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0271e66-be33-4e08-a18f-6b9a35027eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:33.169662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0271e66-be33-4e08-a18f-6b9a35027eac admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.169946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0271e66-be33-4e08-a18f-6b9a35027eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.170252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0271e66-be33-4e08-a18f-6b9a35027eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.171007 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0271e66-be33-4e08-a18f-6b9a35027eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:33.172606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0271e66-be33-4e08-a18f-6b9a35027eac admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:33.174316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4026013629304fec818984e8a829b9c3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.180398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1044/2071] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:33 2022] DELETE /identity/v3/users/4026013629304fec818984e8a829b9c3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:33.185909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1028/2072] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:05:33.186950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:33.188866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:33.256935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f558c3-421f-4c56-846a-4b1af2db04ee admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:33.262436 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac07dc-a19e-45a2-b382-1cf9e3a58366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:33.263348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac07dc-a19e-45a2-b382-1cf9e3a58366 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.263631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac07dc-a19e-45a2-b382-1cf9e3a58366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.263904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac07dc-a19e-45a2-b382-1cf9e3a58366 admin admin] PATH_INFO: `/v3/projects/aaa9e1a61984467ab8efae451b270558/users/4e6db41ec3b94c27bb4af92a35ee0e6d/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.280552 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac07dc-a19e-45a2-b382-1cf9e3a58366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa9e1a61984467ab8efae451b270558, user_id=4e6db41ec3b94c27bb4af92a35ee0e6d, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:33.284193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac07dc-a19e-45a2-b382-1cf9e3a58366 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:33.297980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1045/2073] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:33 2022] PUT /identity/v3/projects/aaa9e1a61984467ab8efae451b270558/users/4e6db41ec3b94c27bb4af92a35ee0e6d/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:33.303025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d72c0-f2b5-47e5-aac7-aab6ee54900c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.303489 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d72c0-f2b5-47e5-aac7-aab6ee54900c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.303796 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d72c0-f2b5-47e5-aac7-aab6ee54900c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.321450 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9b5d72c0-f2b5-47e5-aac7-aab6ee54900c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:33.325152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9b5d72c0-f2b5-47e5-aac7-aab6ee54900c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:33.385492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1029/2074] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:33.402943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac07dc-a19e-45a2-b382-1cf9e3a58366 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:33.404918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-05ac07dc-a19e-45a2-b382-1cf9e3a58366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:33.415046 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:33.462939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d72c0-f2b5-47e5-aac7-aab6ee54900c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:33.464994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9b5d72c0-f2b5-47e5-aac7-aab6ee54900c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:33.467936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac07dc-a19e-45a2-b382-1cf9e3a58366 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:33.471777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dd155-5fb0-4e2f-b531-33e38bfc0b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:33.472446 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd155-5fb0-4e2f-b531-33e38bfc0b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.472766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd155-5fb0-4e2f-b531-33e38bfc0b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.473051 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd155-5fb0-4e2f-b531-33e38bfc0b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.475622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:33.482225 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d72c0-f2b5-47e5-aac7-aab6ee54900c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:33.486045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2dd155-5fb0-4e2f-b531-33e38bfc0b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:33.488317 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2dd155-5fb0-4e2f-b531-33e38bfc0b32 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:33.501478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1046/2075] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:33.506249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:33.507221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.507643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.507930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] PATH_INFO: `/v3/projects/4949621f4e6740a6b5dc86145ee61abe` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.509448 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4949621f4e6740a6b5dc86145ee61abe)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:33.511668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:33.586666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4949621f4e6740a6b5dc86145ee61abe', 'request_id': 'req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.594726 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4949621f4e6740a6b5dc86145ee61abe', 'request_id': 'req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.601227 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4949621f4e6740a6b5dc86145ee61abe', 'request_id': 'req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.620956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] The token cache is being invalidate because project 4949621f4e6740a6b5dc86145ee61abe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:33.621405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.622475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ab6f5745-a2c5-479f-9b36-2b05e065ec6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:33.625926 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1030/2076] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:33 2022] DELETE /identity/v3/projects/4949621f4e6740a6b5dc86145ee61abe => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:33.632532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94370ad-829f-45b7-a572-abed7a43dc00 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.632958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94370ad-829f-45b7-a572-abed7a43dc00 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.633370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94370ad-829f-45b7-a572-abed7a43dc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.678326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a94370ad-829f-45b7-a572-abed7a43dc00 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:33.680452 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a94370ad-829f-45b7-a572-abed7a43dc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:33.758367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1047/2077] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:33.766495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857565c-3303-404e-b801-dfb85deb01f3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.766791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857565c-3303-404e-b801-dfb85deb01f3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.767072 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857565c-3303-404e-b801-dfb85deb01f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.787944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4857565c-3303-404e-b801-dfb85deb01f3 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:33.789481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4857565c-3303-404e-b801-dfb85deb01f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:33.798200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1031/2078] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:33.802701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94370ad-829f-45b7-a572-abed7a43dc00 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:33.814531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94370ad-829f-45b7-a572-abed7a43dc00 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:33.819555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae5234-87a4-4db3-b5c4-aa8588e89dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:33.820398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae5234-87a4-4db3-b5c4-aa8588e89dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.820690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae5234-87a4-4db3-b5c4-aa8588e89dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.820970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae5234-87a4-4db3-b5c4-aa8588e89dc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.821709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae5234-87a4-4db3-b5c4-aa8588e89dc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:33.822021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae5234-87a4-4db3-b5c4-aa8588e89dc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:33.823698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae5234-87a4-4db3-b5c4-aa8588e89dc9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:33.839930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1048/2079] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:05:33.841214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857565c-3303-404e-b801-dfb85deb01f3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:33.850811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857565c-3303-404e-b801-dfb85deb01f3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:33.855754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f6786-7011-4941-9eb1-c4fe0e1df445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:33.856428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f6786-7011-4941-9eb1-c4fe0e1df445 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.856722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f6786-7011-4941-9eb1-c4fe0e1df445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.857000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f6786-7011-4941-9eb1-c4fe0e1df445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.857726 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f6786-7011-4941-9eb1-c4fe0e1df445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:33.859733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f6786-7011-4941-9eb1-c4fe0e1df445 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:33.897230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1032/2080] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:33 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:33.897465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae5234-87a4-4db3-b5c4-aa8588e89dc9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:33.916097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae5234-87a4-4db3-b5c4-aa8588e89dc9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:33.982362 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc09233-8677-4fed-878b-ac5d6e5e590e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:33.983255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc09233-8677-4fed-878b-ac5d6e5e590e admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:33.983542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc09233-8677-4fed-878b-ac5d6e5e590e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:33.983820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc09233-8677-4fed-878b-ac5d6e5e590e admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:33.984712 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc09233-8677-4fed-878b-ac5d6e5e590e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:33.986670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc09233-8677-4fed-878b-ac5d6e5e590e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:34.017024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f6786-7011-4941-9eb1-c4fe0e1df445 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:34.018535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1049/2081] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:33 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:34.035894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f6786-7011-4941-9eb1-c4fe0e1df445 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:34.043177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb34b3-6c5b-403a-b1a9-89b90c0fa313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:34.043863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb34b3-6c5b-403a-b1a9-89b90c0fa313 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:34.044147 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb34b3-6c5b-403a-b1a9-89b90c0fa313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:34.044426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb34b3-6c5b-403a-b1a9-89b90c0fa313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:34.045251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb34b3-6c5b-403a-b1a9-89b90c0fa313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:34.047034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb34b3-6c5b-403a-b1a9-89b90c0fa313 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:34.060340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1033/2082] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:34.065364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc09233-8677-4fed-878b-ac5d6e5e590e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:34.076015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc09233-8677-4fed-878b-ac5d6e5e590e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:34.082494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d25d3d-ae4e-48dd-b818-4fa09aa12dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:34.083191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d25d3d-ae4e-48dd-b818-4fa09aa12dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:34.083482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d25d3d-ae4e-48dd-b818-4fa09aa12dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:34.083759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d25d3d-ae4e-48dd-b818-4fa09aa12dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:34.085050 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d25d3d-ae4e-48dd-b818-4fa09aa12dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:34.088118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d25d3d-ae4e-48dd-b818-4fa09aa12dc0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:34.099082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1050/2083] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:34.103925 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb34b3-6c5b-403a-b1a9-89b90c0fa313 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:34.115257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb34b3-6c5b-403a-b1a9-89b90c0fa313 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:34.120852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e056608-884f-40da-832f-c6f4ea3e19be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:34.121748 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e056608-884f-40da-832f-c6f4ea3e19be admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:34.122030 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e056608-884f-40da-832f-c6f4ea3e19be admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:34.122308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e056608-884f-40da-832f-c6f4ea3e19be admin admin] PATH_INFO: `/v3/projects/4da18592914f4232a9dff93fa0d889a4/users/47efa9b7333e4a14bfe7ea2806553e5c/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:34.139840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e056608-884f-40da-832f-c6f4ea3e19be admin admin] RBAC: Authorizing `identity:create_grant(project_id=4da18592914f4232a9dff93fa0d889a4, user_id=47efa9b7333e4a14bfe7ea2806553e5c, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:34.142521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e056608-884f-40da-832f-c6f4ea3e19be admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:34.163128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1034/2084] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:34 2022] PUT /identity/v3/projects/4da18592914f4232a9dff93fa0d889a4/users/47efa9b7333e4a14bfe7ea2806553e5c/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:34.173978 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d82e27-a51c-4a40-94b1-5490790cb339 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:34.174343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d82e27-a51c-4a40-94b1-5490790cb339 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:34.174743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d82e27-a51c-4a40-94b1-5490790cb339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:34.202193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-57d82e27-a51c-4a40-94b1-5490790cb339 None None] MFA Rules not processed for user `47efa9b7333e4a14bfe7ea2806553e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:34.203629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-57d82e27-a51c-4a40-94b1-5490790cb339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:34.207832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:34.279665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1051/2085] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:34.292057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e056608-884f-40da-832f-c6f4ea3e19be admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:34.293922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4e056608-884f-40da-832f-c6f4ea3e19be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:34.367629 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e056608-884f-40da-832f-c6f4ea3e19be admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:34.371526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a9939-f888-4fa4-9614-7e19427a52f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:34.372275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9939-f888-4fa4-9614-7e19427a52f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:34.372812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9939-f888-4fa4-9614-7e19427a52f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:34.373449 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a9939-f888-4fa4-9614-7e19427a52f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:34.387906 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a9939-f888-4fa4-9614-7e19427a52f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:34.389945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371a9939-f888-4fa4-9614-7e19427a52f3 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:34.401584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1035/2086] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:34.424618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d82e27-a51c-4a40-94b1-5490790cb339 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:34.494902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d82e27-a51c-4a40-94b1-5490790cb339 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:34.499884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:34.500755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:34.501038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:34.501313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] PATH_INFO: `/v3/users/47efa9b7333e4a14bfe7ea2806553e5c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:34.502562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=47efa9b7333e4a14bfe7ea2806553e5c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:34.504487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:34.561430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47efa9b7333e4a14bfe7ea2806553e5c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:34.575018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47efa9b7333e4a14bfe7ea2806553e5c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:34.583715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47efa9b7333e4a14bfe7ea2806553e5c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:34.584570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47efa9b7333e4a14bfe7ea2806553e5c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:34.594874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47efa9b7333e4a14bfe7ea2806553e5c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:34.605188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1052/2087] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:34 2022] DELETE /identity/v3/users/47efa9b7333e4a14bfe7ea2806553e5c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:34.607377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bb009-a0c1-4938-99a7-3a357d35940e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:34.607738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bb009-a0c1-4938-99a7-3a357d35940e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:34.608029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7bb009-a0c1-4938-99a7-3a357d35940e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:34.627016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ef7bb009-a0c1-4938-99a7-3a357d35940e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:34.628247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ef7bb009-a0c1-4938-99a7-3a357d35940e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:34.652625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:34.717679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1036/2088] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:34.719032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:34.721312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:34.784984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:34.845584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48ac53-f987-4d15-bdcd-91f20ef4ca9a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:34.849987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c34008-1c20-4f3c-b210-1749efd5155e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:34.850797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c34008-1c20-4f3c-b210-1749efd5155e service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:34.851198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c34008-1c20-4f3c-b210-1749efd5155e service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:34.851571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c34008-1c20-4f3c-b210-1749efd5155e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:34.862704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c34008-1c20-4f3c-b210-1749efd5155e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:34.864681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c34008-1c20-4f3c-b210-1749efd5155e service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:34.876796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1053/2089] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:35.338829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bb009-a0c1-4938-99a7-3a357d35940e None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:35.340390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ef7bb009-a0c1-4938-99a7-3a357d35940e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:35.379572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7bb009-a0c1-4938-99a7-3a357d35940e None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:35.401591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:35.402703 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:35.403168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:35.403988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] PATH_INFO: `/v3/projects/4da18592914f4232a9dff93fa0d889a4` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:35.410798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4da18592914f4232a9dff93fa0d889a4)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:35.413009 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:35.492153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4da18592914f4232a9dff93fa0d889a4', 'request_id': 'req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:35.511175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4da18592914f4232a9dff93fa0d889a4', 'request_id': 'req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:35.521395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4da18592914f4232a9dff93fa0d889a4', 'request_id': 'req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:35.542837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] The token cache is being invalidate because project 4da18592914f4232a9dff93fa0d889a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:35.543578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:35.544977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c8b7523f-e6c0-47ae-a3c0-ae24313bde4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:35.547756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1037/2090] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:35 2022] DELETE /identity/v3/projects/4da18592914f4232a9dff93fa0d889a4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:35.553007 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc86f9-ed95-4b9a-8ba5-9ebb8d334f86 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:35.553487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc86f9-ed95-4b9a-8ba5-9ebb8d334f86 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:35.553793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc86f9-ed95-4b9a-8ba5-9ebb8d334f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:35.573213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6edc86f9-ed95-4b9a-8ba5-9ebb8d334f86 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:35.574630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6edc86f9-ed95-4b9a-8ba5-9ebb8d334f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:35.659813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1054/2091] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:35.669443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca302b-1c19-4845-85b8-2d8a82dbbe64 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:35.669736 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca302b-1c19-4845-85b8-2d8a82dbbe64 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:35.670021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca302b-1c19-4845-85b8-2d8a82dbbe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:35.695031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-21ca302b-1c19-4845-85b8-2d8a82dbbe64 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:35.696902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-21ca302b-1c19-4845-85b8-2d8a82dbbe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:35.711329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1038/2092] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:35.715625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc86f9-ed95-4b9a-8ba5-9ebb8d334f86 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:35.725987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc86f9-ed95-4b9a-8ba5-9ebb8d334f86 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:35.731031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323734d2-daf2-4774-9dc1-5400c41efdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:35.731830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323734d2-daf2-4774-9dc1-5400c41efdb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:35.732105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323734d2-daf2-4774-9dc1-5400c41efdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:35.732390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323734d2-daf2-4774-9dc1-5400c41efdb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:35.733130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323734d2-daf2-4774-9dc1-5400c41efdb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:35.733443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323734d2-daf2-4774-9dc1-5400c41efdb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:35.735756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323734d2-daf2-4774-9dc1-5400c41efdb7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:35.748367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1055/2093] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:05:35.753079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca302b-1c19-4845-85b8-2d8a82dbbe64 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:35.763049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca302b-1c19-4845-85b8-2d8a82dbbe64 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:35.768358 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bc783-6eca-49a1-a968-abb1204bb91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:35.769167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bc783-6eca-49a1-a968-abb1204bb91d admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:35.769568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bc783-6eca-49a1-a968-abb1204bb91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:35.769888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bc783-6eca-49a1-a968-abb1204bb91d admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:35.771543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bc783-6eca-49a1-a968-abb1204bb91d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:35.773209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bc783-6eca-49a1-a968-abb1204bb91d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:35.804435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1039/2094] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:35 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:35.810021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323734d2-daf2-4774-9dc1-5400c41efdb7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:35.823393 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323734d2-daf2-4774-9dc1-5400c41efdb7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:35.898766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1743e-b497-4484-9f29-643c2b937dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:35.899634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1743e-b497-4484-9f29-643c2b937dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:35.900011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1743e-b497-4484-9f29-643c2b937dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:35.900360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1743e-b497-4484-9f29-643c2b937dcc admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:35.901282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1743e-b497-4484-9f29-643c2b937dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:35.903463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1743e-b497-4484-9f29-643c2b937dcc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:35.932781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1056/2095] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:35 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:05:35.937947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bc783-6eca-49a1-a968-abb1204bb91d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:35.951553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bc783-6eca-49a1-a968-abb1204bb91d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:35.958880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc42cfe-0b9a-4429-9448-11681c755735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:35.960196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc42cfe-0b9a-4429-9448-11681c755735 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:35.960670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc42cfe-0b9a-4429-9448-11681c755735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:35.961093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc42cfe-0b9a-4429-9448-11681c755735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:35.961968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc42cfe-0b9a-4429-9448-11681c755735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:35.963874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc42cfe-0b9a-4429-9448-11681c755735 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:35.983050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1040/2096] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:35.988070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1743e-b497-4484-9f29-643c2b937dcc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:36.001003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1743e-b497-4484-9f29-643c2b937dcc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:36.005980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb6ded8-16ae-40bc-90b0-0a0c3ca0190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:36.006683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6ded8-16ae-40bc-90b0-0a0c3ca0190d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:36.006967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6ded8-16ae-40bc-90b0-0a0c3ca0190d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:36.007795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb6ded8-16ae-40bc-90b0-0a0c3ca0190d admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:36.007996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6ded8-16ae-40bc-90b0-0a0c3ca0190d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:36.009845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb6ded8-16ae-40bc-90b0-0a0c3ca0190d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:36.025707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1057/2097] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:36.027362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc42cfe-0b9a-4429-9448-11681c755735 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:36.039394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc42cfe-0b9a-4429-9448-11681c755735 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:36.063389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf869389-3035-470b-888f-b495d01147e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:36.064303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf869389-3035-470b-888f-b495d01147e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:36.064581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf869389-3035-470b-888f-b495d01147e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:36.064855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf869389-3035-470b-888f-b495d01147e4 admin admin] PATH_INFO: `/v3/projects/24abfcce8fb74bd4845f211d4dab0f76/users/0a1a04346f634937a8683eaeaa46e807/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:36.098262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf869389-3035-470b-888f-b495d01147e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24abfcce8fb74bd4845f211d4dab0f76, user_id=0a1a04346f634937a8683eaeaa46e807, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:36.101101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf869389-3035-470b-888f-b495d01147e4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:36.130127 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1041/2098] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:36 2022] PUT /identity/v3/projects/24abfcce8fb74bd4845f211d4dab0f76/users/0a1a04346f634937a8683eaeaa46e807/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:36.141041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45511-c690-46ae-bd49-117815a3884f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:36.141335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45511-c690-46ae-bd49-117815a3884f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:36.141615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f45511-c690-46ae-bd49-117815a3884f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:36.199852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-15f45511-c690-46ae-bd49-117815a3884f None None] MFA Rules not processed for user `0a1a04346f634937a8683eaeaa46e807`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:36.203659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-15f45511-c690-46ae-bd49-117815a3884f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:36.244956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:36.303937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1058/2099] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:36.315149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf869389-3035-470b-888f-b495d01147e4 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:36.316962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bf869389-3035-470b-888f-b495d01147e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:36.367299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:36.382112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf869389-3035-470b-888f-b495d01147e4 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:36.387702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe3d8c5-6100-4f88-89ca-fa2f927d4197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:36.388508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3d8c5-6100-4f88-89ca-fa2f927d4197 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:36.390006 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3d8c5-6100-4f88-89ca-fa2f927d4197 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:36.390577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe3d8c5-6100-4f88-89ca-fa2f927d4197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:36.401595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3d8c5-6100-4f88-89ca-fa2f927d4197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:36.403148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe3d8c5-6100-4f88-89ca-fa2f927d4197 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:36.413984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1042/2100] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:38.469022 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f45511-c690-46ae-bd49-117815a3884f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:38.471134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-15f45511-c690-46ae-bd49-117815a3884f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:38.675691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f45511-c690-46ae-bd49-117815a3884f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:38.681285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:38.682554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:38.683047 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:38.683524 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] PATH_INFO: `/v3/users/24c21881b22e40878a69689b63155484` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:38.685038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c21881b22e40878a69689b63155484)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:38.687933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:38.767763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c21881b22e40878a69689b63155484'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:38.787439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c21881b22e40878a69689b63155484'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:38.798163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c21881b22e40878a69689b63155484'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:38.798878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c21881b22e40878a69689b63155484'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:38.810259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c21881b22e40878a69689b63155484'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:38.823423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1059/2101] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:38 2022] DELETE /identity/v3/users/24c21881b22e40878a69689b63155484 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:38.824168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871462d5-1838-4c5f-b914-e46504c1e53e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:38.824650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871462d5-1838-4c5f-b914-e46504c1e53e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:38.824731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871462d5-1838-4c5f-b914-e46504c1e53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:38.844268 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-871462d5-1838-4c5f-b914-e46504c1e53e None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:38.845454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-871462d5-1838-4c5f-b914-e46504c1e53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:38.928081 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1043/2102] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:38.938637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:38.942115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:39.023416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:39.047062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5eb904-3608-4b53-8ac7-4483b92fafb8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:39.056108 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f7a22-f2ce-4c55-8b3c-062e8043e449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:39.058061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f7a22-f2ce-4c55-8b3c-062e8043e449 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:39.058358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f7a22-f2ce-4c55-8b3c-062e8043e449 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:39.058667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f7a22-f2ce-4c55-8b3c-062e8043e449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:39.073843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f7a22-f2ce-4c55-8b3c-062e8043e449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:39.075377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f7a22-f2ce-4c55-8b3c-062e8043e449 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:39.094720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1060/2103] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:39.272246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2aa43-06ee-487b-892c-13894e5e5dda None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:39.272536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2aa43-06ee-487b-892c-13894e5e5dda None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:39.272819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2aa43-06ee-487b-892c-13894e5e5dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:39.303813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b2a2aa43-06ee-487b-892c-13894e5e5dda None None] MFA Rules not processed for user `4e6db41ec3b94c27bb4af92a35ee0e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:39.305032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b2a2aa43-06ee-487b-892c-13894e5e5dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:39.385094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1044/2104] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:39.400131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f7a22-f2ce-4c55-8b3c-062e8043e449 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:39.401619 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0c4f7a22-f2ce-4c55-8b3c-062e8043e449 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:39.481497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f7a22-f2ce-4c55-8b3c-062e8043e449 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:39.488954 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76c44-d831-45c3-8f2e-c4f239f533f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:39.489790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76c44-d831-45c3-8f2e-c4f239f533f5 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:39.489994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76c44-d831-45c3-8f2e-c4f239f533f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:39.490244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76c44-d831-45c3-8f2e-c4f239f533f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:39.498149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2aa43-06ee-487b-892c-13894e5e5dda None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:39.500639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c76c44-d831-45c3-8f2e-c4f239f533f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:39.501222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b2a2aa43-06ee-487b-892c-13894e5e5dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:39.502260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c76c44-d831-45c3-8f2e-c4f239f533f5 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:39.514834 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1061/2105] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:39.524570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2aa43-06ee-487b-892c-13894e5e5dda None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:39.535727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:39.536696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:39.537075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:39.537472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] PATH_INFO: `/v3/projects/2edda53a2da04945bf3d7b7ea60de3a0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:39.538950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2edda53a2da04945bf3d7b7ea60de3a0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:39.541132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:39.625682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2edda53a2da04945bf3d7b7ea60de3a0', 'request_id': 'req-727afb71-5bea-4535-9ac8-29f78434d659'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:39.638918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2edda53a2da04945bf3d7b7ea60de3a0', 'request_id': 'req-727afb71-5bea-4535-9ac8-29f78434d659'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:39.663096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2edda53a2da04945bf3d7b7ea60de3a0', 'request_id': 'req-727afb71-5bea-4535-9ac8-29f78434d659'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:39.678326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:39.684152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] The token cache is being invalidate because project 2edda53a2da04945bf3d7b7ea60de3a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:39.684915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:39.686744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:39.693331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1045/2106] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:39 2022] DELETE /identity/v3/projects/2edda53a2da04945bf3d7b7ea60de3a0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:39.699278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76c44-d831-45c3-8f2e-c4f239f533f5 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:39.716678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76c44-d831-45c3-8f2e-c4f239f533f5 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:39.839317 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:39.842002 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:39.842365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:39.842674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] PATH_INFO: `/v3/users/df840ff1e4e348838074ee6c39e67afe` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:39.846958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df840ff1e4e348838074ee6c39e67afe)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:39.853198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:39.958917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df840ff1e4e348838074ee6c39e67afe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:39.996357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df840ff1e4e348838074ee6c39e67afe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:40.011815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df840ff1e4e348838074ee6c39e67afe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:40.012785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df840ff1e4e348838074ee6c39e67afe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:40.014544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:40.025123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-235bfef1-da40-4458-9553-2e617d8dc748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df840ff1e4e348838074ee6c39e67afe'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:40.042941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1062/2107] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:39 2022] DELETE /identity/v3/users/df840ff1e4e348838074ee6c39e67afe => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:40.431313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:40.432778 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:40.450639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727afb71-5bea-4535-9ac8-29f78434d659 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:40.526619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:40.527497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:40.527793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:40.528078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] PATH_INFO: `/v3/projects/df273ffbc54c46f0ab8d9daffbfa0a4f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:40.529331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df273ffbc54c46f0ab8d9daffbfa0a4f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:40.531512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:40.585702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df273ffbc54c46f0ab8d9daffbfa0a4f', 'request_id': 'req-abe350b6-19f7-48b2-b0f2-ff4310435bf7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:40.592066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df273ffbc54c46f0ab8d9daffbfa0a4f', 'request_id': 'req-abe350b6-19f7-48b2-b0f2-ff4310435bf7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:40.600022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df273ffbc54c46f0ab8d9daffbfa0a4f', 'request_id': 'req-abe350b6-19f7-48b2-b0f2-ff4310435bf7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:40.614033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] The token cache is being invalidate because project df273ffbc54c46f0ab8d9daffbfa0a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:40.614340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:40.615213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-abe350b6-19f7-48b2-b0f2-ff4310435bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:40.620520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1046/2108] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:40 2022] DELETE /identity/v3/projects/df273ffbc54c46f0ab8d9daffbfa0a4f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:40.623360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c991b-66ea-4b4b-ac33-197e505265b7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:40.624858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c991b-66ea-4b4b-ac33-197e505265b7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:40.624993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c991b-66ea-4b4b-ac33-197e505265b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:40.651793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bf7c991b-66ea-4b4b-ac33-197e505265b7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:40.653246 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bf7c991b-66ea-4b4b-ac33-197e505265b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:40.732146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1063/2109] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:40.739872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469504a9-d596-4673-b238-a941ba711cdc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:40.740472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469504a9-d596-4673-b238-a941ba711cdc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:40.740762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469504a9-d596-4673-b238-a941ba711cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:40.760686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-469504a9-d596-4673-b238-a941ba711cdc None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:40.762135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-469504a9-d596-4673-b238-a941ba711cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:40.772349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1047/2110] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:40.776388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c991b-66ea-4b4b-ac33-197e505265b7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:40.786519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c991b-66ea-4b4b-ac33-197e505265b7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:40.792993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726cf37-0f97-4217-a474-770a29f093e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:40.793573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726cf37-0f97-4217-a474-770a29f093e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:40.793864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726cf37-0f97-4217-a474-770a29f093e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:40.794273 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726cf37-0f97-4217-a474-770a29f093e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:40.795183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726cf37-0f97-4217-a474-770a29f093e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:05:40.795498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726cf37-0f97-4217-a474-770a29f093e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:40.797158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726cf37-0f97-4217-a474-770a29f093e4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:40.828615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1064/2111] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:05:40.836263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469504a9-d596-4673-b238-a941ba711cdc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:40.847387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469504a9-d596-4673-b238-a941ba711cdc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:40.852815 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233e4b8-d6b6-431d-a861-96306e3c2619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:40.853620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233e4b8-d6b6-431d-a861-96306e3c2619 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:40.854403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233e4b8-d6b6-431d-a861-96306e3c2619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:40.854403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233e4b8-d6b6-431d-a861-96306e3c2619 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:40.855094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233e4b8-d6b6-431d-a861-96306e3c2619 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:40.857069 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233e4b8-d6b6-431d-a861-96306e3c2619 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:40.888179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1048/2112] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:40 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:40.892914 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726cf37-0f97-4217-a474-770a29f093e4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:40.911181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726cf37-0f97-4217-a474-770a29f093e4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:40.984274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05feb73e-96b9-41fa-b598-b62d40935a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:40.985081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05feb73e-96b9-41fa-b598-b62d40935a14 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:40.985378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05feb73e-96b9-41fa-b598-b62d40935a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:40.985661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05feb73e-96b9-41fa-b598-b62d40935a14 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:40.986369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05feb73e-96b9-41fa-b598-b62d40935a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:40.988649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05feb73e-96b9-41fa-b598-b62d40935a14 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:41.023384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1065/2113] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:40 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:41.027768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233e4b8-d6b6-431d-a861-96306e3c2619 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:41.045077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233e4b8-d6b6-431d-a861-96306e3c2619 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:41.054543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249aea6-5c3d-4af5-b1ad-7bd554cc6de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:41.055249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249aea6-5c3d-4af5-b1ad-7bd554cc6de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:41.055597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249aea6-5c3d-4af5-b1ad-7bd554cc6de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:41.055873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249aea6-5c3d-4af5-b1ad-7bd554cc6de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:41.056686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249aea6-5c3d-4af5-b1ad-7bd554cc6de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:41.059033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249aea6-5c3d-4af5-b1ad-7bd554cc6de9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:41.073625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1049/2114] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:41.077411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05feb73e-96b9-41fa-b598-b62d40935a14 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:41.087700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05feb73e-96b9-41fa-b598-b62d40935a14 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:41.093470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9f676-ba8a-4eff-b503-f39313b7ffa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:41.093887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9f676-ba8a-4eff-b503-f39313b7ffa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:41.094173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9f676-ba8a-4eff-b503-f39313b7ffa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:41.094640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9f676-ba8a-4eff-b503-f39313b7ffa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:41.096320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9f676-ba8a-4eff-b503-f39313b7ffa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:41.098305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe9f676-ba8a-4eff-b503-f39313b7ffa1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:41.115511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249aea6-5c3d-4af5-b1ad-7bd554cc6de9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:41.132571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1066/2115] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:41.142344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249aea6-5c3d-4af5-b1ad-7bd554cc6de9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:41.153016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfb937-9363-4e12-9b41-439657b01f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:41.153937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfb937-9363-4e12-9b41-439657b01f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:41.154231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfb937-9363-4e12-9b41-439657b01f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:41.154546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfb937-9363-4e12-9b41-439657b01f63 admin admin] PATH_INFO: `/v3/projects/7087555844ad4fd5b1aeae567db03082/users/d3e1a5cc5a10485dbc9609c5e7c4d2db/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:41.186210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfb937-9363-4e12-9b41-439657b01f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7087555844ad4fd5b1aeae567db03082, user_id=d3e1a5cc5a10485dbc9609c5e7c4d2db, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:41.189868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfb937-9363-4e12-9b41-439657b01f63 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:41.211874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1050/2116] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:41 2022] PUT /identity/v3/projects/7087555844ad4fd5b1aeae567db03082/users/d3e1a5cc5a10485dbc9609c5e7c4d2db/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:41.224679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391b6b2-568c-4d33-ae7b-9bf895cafc8c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:41.225848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391b6b2-568c-4d33-ae7b-9bf895cafc8c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:41.226672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391b6b2-568c-4d33-ae7b-9bf895cafc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:41.297872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a391b6b2-568c-4d33-ae7b-9bf895cafc8c None None] MFA Rules not processed for user `d3e1a5cc5a10485dbc9609c5e7c4d2db`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:41.299665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a391b6b2-568c-4d33-ae7b-9bf895cafc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:41.402087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:41.465454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfb937-9363-4e12-9b41-439657b01f63 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:41.467820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1067/2117] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:41.510746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:41.559251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3bcea-092e-4bb5-88c7-185ac53572f4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:41.559562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3bcea-092e-4bb5-88c7-185ac53572f4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:41.559817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3bcea-092e-4bb5-88c7-185ac53572f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:41.570418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfb937-9363-4e12-9b41-439657b01f63 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:41.578491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced6272-e8be-4f30-be04-9839f2e9acf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:41.579679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced6272-e8be-4f30-be04-9839f2e9acf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:41.580260 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced6272-e8be-4f30-be04-9839f2e9acf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:41.580385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c5d3bcea-092e-4bb5-88c7-185ac53572f4 None None] MFA Rules not processed for user `4e6db41ec3b94c27bb4af92a35ee0e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:41.580856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced6272-e8be-4f30-be04-9839f2e9acf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:41.581754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced6272-e8be-4f30-be04-9839f2e9acf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:41.584158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced6272-e8be-4f30-be04-9839f2e9acf3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:41.585606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c5d3bcea-092e-4bb5-88c7-185ac53572f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:41.631124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1051/2118] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:05:41 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:41.638482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced6272-e8be-4f30-be04-9839f2e9acf3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:41.739743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced6272-e8be-4f30-be04-9839f2e9acf3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:41.746274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0320ca-dc29-4cb8-8f3e-1c54da20e23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:41.746986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0320ca-dc29-4cb8-8f3e-1c54da20e23a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:41.748139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0320ca-dc29-4cb8-8f3e-1c54da20e23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:41.748440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0320ca-dc29-4cb8-8f3e-1c54da20e23a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:41.749140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0320ca-dc29-4cb8-8f3e-1c54da20e23a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:41.752270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0320ca-dc29-4cb8-8f3e-1c54da20e23a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:41.780394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1068/2119] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:05:41.822791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1052/2120] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:05:41 2022] POST /identity/v3/users => generated 295 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:05:41.825226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3bcea-092e-4bb5-88c7-185ac53572f4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:41.862838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3bcea-092e-4bb5-88c7-185ac53572f4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:41.891468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981cf29-a495-44d1-a14a-b4579f683817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:41.892253 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981cf29-a495-44d1-a14a-b4579f683817 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:41.892545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981cf29-a495-44d1-a14a-b4579f683817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:41.892827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981cf29-a495-44d1-a14a-b4579f683817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:41.893838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981cf29-a495-44d1-a14a-b4579f683817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:41.897239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4981cf29-a495-44d1-a14a-b4579f683817 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:41.924041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1069/2121] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:41.925878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0320ca-dc29-4cb8-8f3e-1c54da20e23a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:41.941676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0320ca-dc29-4cb8-8f3e-1c54da20e23a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:41.951799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c654c75-23a1-49b6-abf3-01963914d128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:41.960208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c654c75-23a1-49b6-abf3-01963914d128 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:41.960878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c654c75-23a1-49b6-abf3-01963914d128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:41.961388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c654c75-23a1-49b6-abf3-01963914d128 admin admin] PATH_INFO: `/v3/projects/c3b8516383b243efaf77a4f7319859ce/users/1a636dfa88444a079f42d0c33ae1d47a/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:41.980082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c654c75-23a1-49b6-abf3-01963914d128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b8516383b243efaf77a4f7319859ce, user_id=1a636dfa88444a079f42d0c33ae1d47a, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:41.982912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c654c75-23a1-49b6-abf3-01963914d128 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:42.002107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1053/2122] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:41 2022] PUT /identity/v3/projects/c3b8516383b243efaf77a4f7319859ce/users/1a636dfa88444a079f42d0c33ae1d47a/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:42.017126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981cf29-a495-44d1-a14a-b4579f683817 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:42.040738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4981cf29-a495-44d1-a14a-b4579f683817 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:42.153429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333a079-992b-48fb-806c-276fc39624e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:42.154254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333a079-992b-48fb-806c-276fc39624e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:42.156299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333a079-992b-48fb-806c-276fc39624e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:42.156701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333a079-992b-48fb-806c-276fc39624e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:42.158201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333a079-992b-48fb-806c-276fc39624e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:42.160961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333a079-992b-48fb-806c-276fc39624e2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:42.173945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1070/2123] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:05:42.177921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c654c75-23a1-49b6-abf3-01963914d128 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:42.195682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c654c75-23a1-49b6-abf3-01963914d128 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:42.211227 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9242df-312b-4ca1-be1f-35a4d23db5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:42.212126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9242df-312b-4ca1-be1f-35a4d23db5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:42.222603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9242df-312b-4ca1-be1f-35a4d23db5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:42.222708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9242df-312b-4ca1-be1f-35a4d23db5a3 admin admin] PATH_INFO: `/v3/projects/c3b8516383b243efaf77a4f7319859ce/users/1a636dfa88444a079f42d0c33ae1d47a/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:42.227088 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9242df-312b-4ca1-be1f-35a4d23db5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3b8516383b243efaf77a4f7319859ce, user_id=1a636dfa88444a079f42d0c33ae1d47a, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:42.230266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9242df-312b-4ca1-be1f-35a4d23db5a3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:42.252855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1054/2124] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:05:42 2022] PUT /identity/v3/projects/c3b8516383b243efaf77a4f7319859ce/users/1a636dfa88444a079f42d0c33ae1d47a/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:05:42.263908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5394-da9e-44ca-a5c6-da1f1f847abd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:42.264206 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5394-da9e-44ca-a5c6-da1f1f847abd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:42.265543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5394-da9e-44ca-a5c6-da1f1f847abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:42.300553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5bee5394-da9e-44ca-a5c6-da1f1f847abd None None] MFA Rules not processed for user `1a636dfa88444a079f42d0c33ae1d47a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:42.302700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5bee5394-da9e-44ca-a5c6-da1f1f847abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:42.409331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:42.414044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1071/2125] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:42.427699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9242df-312b-4ca1-be1f-35a4d23db5a3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:42.434286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fe9242df-312b-4ca1-be1f-35a4d23db5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:42.507048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9242df-312b-4ca1-be1f-35a4d23db5a3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:42.511151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8677136-8f69-4d91-ab73-6d88605b6fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:42.512368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8677136-8f69-4d91-ab73-6d88605b6fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:42.512639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8677136-8f69-4d91-ab73-6d88605b6fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:42.513095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8677136-8f69-4d91-ab73-6d88605b6fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:42.535625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:42.607278 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8677136-8f69-4d91-ab73-6d88605b6fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:42.615137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8677136-8f69-4d91-ab73-6d88605b6fe0 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:42.631519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1055/2126] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:48.532066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee5394-da9e-44ca-a5c6-da1f1f847abd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:48.533557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5bee5394-da9e-44ca-a5c6-da1f1f847abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:48.545334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee5394-da9e-44ca-a5c6-da1f1f847abd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:48.549335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38aed3-bcb9-417a-b5a2-af573dabaeae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:48.549973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38aed3-bcb9-417a-b5a2-af573dabaeae service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:48.550262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38aed3-bcb9-417a-b5a2-af573dabaeae service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:48.550722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38aed3-bcb9-417a-b5a2-af573dabaeae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:48.559667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38aed3-bcb9-417a-b5a2-af573dabaeae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:48.561209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38aed3-bcb9-417a-b5a2-af573dabaeae service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:48.572110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1072/2127] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:52.624827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=19 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:53.291580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8677136-8f69-4d91-ab73-6d88605b6fe0 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:53.293087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a8677136-8f69-4d91-ab73-6d88605b6fe0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:53.362387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8677136-8f69-4d91-ab73-6d88605b6fe0 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:53.367531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:53.368398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:53.368680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:53.368954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] PATH_INFO: `/v3/users/0a1a04346f634937a8683eaeaa46e807` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:53.370490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a1a04346f634937a8683eaeaa46e807)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:53.372134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:53.430745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a1a04346f634937a8683eaeaa46e807'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:53.431402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a1a04346f634937a8683eaeaa46e807'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:53.438365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a1a04346f634937a8683eaeaa46e807'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:53.439079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a1a04346f634937a8683eaeaa46e807'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:53.451040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a1a04346f634937a8683eaeaa46e807'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:53.462986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1056/2128] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:05:53 2022] DELETE /identity/v3/users/0a1a04346f634937a8683eaeaa46e807 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:53.468783 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c105a-8d71-47c1-b83c-82f685d8642a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:53.469378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c105a-8d71-47c1-b83c-82f685d8642a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:53.469498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0c105a-8d71-47c1-b83c-82f685d8642a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:53.488097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7d0c105a-8d71-47c1-b83c-82f685d8642a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:53.489285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7d0c105a-8d71-47c1-b83c-82f685d8642a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:53.549765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1073/2129] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:05:53.559179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:53.560564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:53.617518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d09240-a103-491d-9d56-44d53a0bc11d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:53.621181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61adbb-8e07-4573-a151-8bfc0c36567a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:53.621819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61adbb-8e07-4573-a151-8bfc0c36567a service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:53.622103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61adbb-8e07-4573-a151-8bfc0c36567a service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:53.622383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61adbb-8e07-4573-a151-8bfc0c36567a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:53.631652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61adbb-8e07-4573-a151-8bfc0c36567a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:53.633090 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61adbb-8e07-4573-a151-8bfc0c36567a service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:53.643420 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1057/2130] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:05:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:05:53.901833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c105a-8d71-47c1-b83c-82f685d8642a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:05:53.902905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7d0c105a-8d71-47c1-b83c-82f685d8642a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:53.924841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0c105a-8d71-47c1-b83c-82f685d8642a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:05:53.932680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:05:53.933565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:53.933779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:53.934044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] PATH_INFO: `/v3/projects/24abfcce8fb74bd4845f211d4dab0f76` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:53.935382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24abfcce8fb74bd4845f211d4dab0f76)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:05:53.937872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:05:53.986913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24abfcce8fb74bd4845f211d4dab0f76', 'request_id': 'req-72d2c882-6955-432b-aa8f-5b3cfefd4298'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:53.993189 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24abfcce8fb74bd4845f211d4dab0f76', 'request_id': 'req-72d2c882-6955-432b-aa8f-5b3cfefd4298'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:53.999634 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24abfcce8fb74bd4845f211d4dab0f76', 'request_id': 'req-72d2c882-6955-432b-aa8f-5b3cfefd4298'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:54.012391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] The token cache is being invalidate because project 24abfcce8fb74bd4845f211d4dab0f76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:05:54.012680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:54.013294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:05:54.015442 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1074/2131] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:05:53 2022] DELETE /identity/v3/projects/24abfcce8fb74bd4845f211d4dab0f76 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:05:58.336591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b4f9d-b867-4788-9f17-743291452f1c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:05:58.337084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b4f9d-b867-4788-9f17-743291452f1c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:05:58.337180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b4f9d-b867-4788-9f17-743291452f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:05:58.353920 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-861b4f9d-b867-4788-9f17-743291452f1c None None] MFA Rules not processed for user `4a3f5eeb2f01498aa4c84c5ecc6a6cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:05:58.355140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-861b4f9d-b867-4788-9f17-743291452f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:05:58.371588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:05:58.418534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1058/2132] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:04.009433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:08.408899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=15 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:13.759289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:13.760742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:13.837836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2c882-6955-432b-aa8f-5b3cfefd4298 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:13.843048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:13.843929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:13.844220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:13.844504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] PATH_INFO: `/v3/users/d3e1a5cc5a10485dbc9609c5e7c4d2db` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:13.845717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3e1a5cc5a10485dbc9609c5e7c4d2db)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:13.847704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:13.907453 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3e1a5cc5a10485dbc9609c5e7c4d2db'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:13.919729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3e1a5cc5a10485dbc9609c5e7c4d2db'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:13.927042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3e1a5cc5a10485dbc9609c5e7c4d2db'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:13.927661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3e1a5cc5a10485dbc9609c5e7c4d2db'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:13.934684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3e1a5cc5a10485dbc9609c5e7c4d2db'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:13.941243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1075/2133] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:13 2022] DELETE /identity/v3/users/d3e1a5cc5a10485dbc9609c5e7c4d2db => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:13.945709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b513a-5978-4c42-aa21-01a0f32bd62b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:13.946176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b513a-5978-4c42-aa21-01a0f32bd62b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:13.946305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b513a-5978-4c42-aa21-01a0f32bd62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:13.964549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-591b513a-5978-4c42-aa21-01a0f32bd62b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:13.966124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-591b513a-5978-4c42-aa21-01a0f32bd62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:14.028741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1059/2134] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:06:14.038707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:14.040090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:14.097877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24541c42-d95b-471a-82df-646c5bd21eb2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:14.101528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605761d-7ddb-408d-8061-0ea3609d4012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:14.102170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605761d-7ddb-408d-8061-0ea3609d4012 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:14.102469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605761d-7ddb-408d-8061-0ea3609d4012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:14.102755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605761d-7ddb-408d-8061-0ea3609d4012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:14.111805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605761d-7ddb-408d-8061-0ea3609d4012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:14.113220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605761d-7ddb-408d-8061-0ea3609d4012 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:14.122799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1076/2135] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:06:14.386950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b513a-5978-4c42-aa21-01a0f32bd62b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:14.388580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-591b513a-5978-4c42-aa21-01a0f32bd62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:14.401911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b513a-5978-4c42-aa21-01a0f32bd62b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:14.406561 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:14.407464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:14.407749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:14.408024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] PATH_INFO: `/v3/projects/7087555844ad4fd5b1aeae567db03082` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:14.409198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7087555844ad4fd5b1aeae567db03082)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:14.411212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:14.457539 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7087555844ad4fd5b1aeae567db03082', 'request_id': 'req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:14.463138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7087555844ad4fd5b1aeae567db03082', 'request_id': 'req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:14.469278 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7087555844ad4fd5b1aeae567db03082', 'request_id': 'req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:14.484376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] The token cache is being invalidate because project 7087555844ad4fd5b1aeae567db03082 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:14.484376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:14.484661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:14.486122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1060/2136] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:14 2022] DELETE /identity/v3/projects/7087555844ad4fd5b1aeae567db03082 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:14.491244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605761d-7ddb-408d-8061-0ea3609d4012 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:14.501926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605761d-7ddb-408d-8061-0ea3609d4012 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:14.520157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:14.559333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:14.560226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:14.560512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:14.561310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] PATH_INFO: `/v3/users/1a636dfa88444a079f42d0c33ae1d47a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:14.562545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a636dfa88444a079f42d0c33ae1d47a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:14.564577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:14.609331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a636dfa88444a079f42d0c33ae1d47a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:14.620879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a636dfa88444a079f42d0c33ae1d47a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:14.629281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a636dfa88444a079f42d0c33ae1d47a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:14.630305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a636dfa88444a079f42d0c33ae1d47a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:14.637341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ba038c54-7a0b-4d66-8804-f9f268886ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a636dfa88444a079f42d0c33ae1d47a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:14.644013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1077/2137] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:14 2022] DELETE /identity/v3/users/1a636dfa88444a079f42d0c33ae1d47a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:14.914890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:14.916352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:14.929751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b6924-a6b6-4c3c-8fdf-58ebe710c128 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:14.988945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:14.989952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:14.990234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:14.990550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] PATH_INFO: `/v3/projects/c3b8516383b243efaf77a4f7319859ce` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:14.992200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3b8516383b243efaf77a4f7319859ce)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:14.993825 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:15.031456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:15.057086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3b8516383b243efaf77a4f7319859ce', 'request_id': 'req-00664b7a-7bab-4986-b441-d9fdaad75017'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:15.062724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3b8516383b243efaf77a4f7319859ce', 'request_id': 'req-00664b7a-7bab-4986-b441-d9fdaad75017'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:15.069169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3b8516383b243efaf77a4f7319859ce', 'request_id': 'req-00664b7a-7bab-4986-b441-d9fdaad75017'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:15.081970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] The token cache is being invalidate because project c3b8516383b243efaf77a4f7319859ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:15.082268 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:15.082956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-00664b7a-7bab-4986-b441-d9fdaad75017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:15.084510 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1061/2138] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:14 2022] DELETE /identity/v3/projects/c3b8516383b243efaf77a4f7319859ce => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:15.089536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f1a37-6d31-456a-a742-8ad530e51538 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.090010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f1a37-6d31-456a-a742-8ad530e51538 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.090302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f1a37-6d31-456a-a742-8ad530e51538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.109263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-128f1a37-6d31-456a-a742-8ad530e51538 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:15.110578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-128f1a37-6d31-456a-a742-8ad530e51538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:15.174116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1078/2139] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:06:15.190805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065fe12-5dd3-4bf9-8d96-aa485f728206 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.190982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065fe12-5dd3-4bf9-8d96-aa485f728206 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.191175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065fe12-5dd3-4bf9-8d96-aa485f728206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.219448 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9065fe12-5dd3-4bf9-8d96-aa485f728206 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:15.220712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9065fe12-5dd3-4bf9-8d96-aa485f728206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:15.238091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1062/2140] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:15.244408 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f1a37-6d31-456a-a742-8ad530e51538 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:15.257095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f1a37-6d31-456a-a742-8ad530e51538 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:15.261778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ff4a9-2dd7-419a-8151-3759cc77cdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:15.262519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ff4a9-2dd7-419a-8151-3759cc77cdd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.262761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ff4a9-2dd7-419a-8151-3759cc77cdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.263038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ff4a9-2dd7-419a-8151-3759cc77cdd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.263761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ff4a9-2dd7-419a-8151-3759cc77cdd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:15.264717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ff4a9-2dd7-419a-8151-3759cc77cdd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:15.266321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ff4a9-2dd7-419a-8151-3759cc77cdd3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:15.278840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1079/2141] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:15.283216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065fe12-5dd3-4bf9-8d96-aa485f728206 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:15.292306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065fe12-5dd3-4bf9-8d96-aa485f728206 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:15.296976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637a331-e72b-4819-a847-e054786c7b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:15.297620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637a331-e72b-4819-a847-e054786c7b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.298297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637a331-e72b-4819-a847-e054786c7b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.298297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1637a331-e72b-4819-a847-e054786c7b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.298895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637a331-e72b-4819-a847-e054786c7b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:15.300799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1637a331-e72b-4819-a847-e054786c7b42 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:15.328845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1063/2142] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:06:15 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:15.332602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ff4a9-2dd7-419a-8151-3759cc77cdd3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:15.343485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ff4a9-2dd7-419a-8151-3759cc77cdd3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:15.371515 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:15.404575 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860965c-68d2-44d2-a97d-917d09030615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:15.405243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860965c-68d2-44d2-a97d-917d09030615 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.405527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860965c-68d2-44d2-a97d-917d09030615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.405801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860965c-68d2-44d2-a97d-917d09030615 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.406507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860965c-68d2-44d2-a97d-917d09030615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:15.408421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860965c-68d2-44d2-a97d-917d09030615 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:15.429107 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1080/2143] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:06:15 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:15.433230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637a331-e72b-4819-a847-e054786c7b42 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:15.444271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1637a331-e72b-4819-a847-e054786c7b42 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:15.449036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70514c-c583-4f88-8091-5464635bb213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:15.449681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70514c-c583-4f88-8091-5464635bb213 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.449959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70514c-c583-4f88-8091-5464635bb213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.450234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70514c-c583-4f88-8091-5464635bb213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.451012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70514c-c583-4f88-8091-5464635bb213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:15.452620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70514c-c583-4f88-8091-5464635bb213 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:15.463068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1064/2144] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:15.466829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860965c-68d2-44d2-a97d-917d09030615 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:15.476781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860965c-68d2-44d2-a97d-917d09030615 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:15.481412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a811e-5f8b-4937-9492-fba6aa1a87b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:15.482060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a811e-5f8b-4937-9492-fba6aa1a87b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.482367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a811e-5f8b-4937-9492-fba6aa1a87b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.483244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a811e-5f8b-4937-9492-fba6aa1a87b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.484095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a811e-5f8b-4937-9492-fba6aa1a87b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:15.485708 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a811e-5f8b-4937-9492-fba6aa1a87b9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:15.495424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1081/2145] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:15.500150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70514c-c583-4f88-8091-5464635bb213 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:15.509750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70514c-c583-4f88-8091-5464635bb213 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:15.514816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea666e-4b44-4dff-aaa2-4162a0596aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:15.515671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea666e-4b44-4dff-aaa2-4162a0596aac admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.515948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea666e-4b44-4dff-aaa2-4162a0596aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.516219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea666e-4b44-4dff-aaa2-4162a0596aac admin admin] PATH_INFO: `/v3/projects/881780ecfb4d4b0f871f9e79bed72e3b/users/1063122c1462470fa74b40a6d43e0ac4/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.530963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea666e-4b44-4dff-aaa2-4162a0596aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=881780ecfb4d4b0f871f9e79bed72e3b, user_id=1063122c1462470fa74b40a6d43e0ac4, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:15.533645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea666e-4b44-4dff-aaa2-4162a0596aac admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:15.549337 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1065/2146] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:15 2022] PUT /identity/v3/projects/881780ecfb4d4b0f871f9e79bed72e3b/users/1063122c1462470fa74b40a6d43e0ac4/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:15.563088 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398633-cf32-4dfd-b5f8-b17cbdf7f9d8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.563377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398633-cf32-4dfd-b5f8-b17cbdf7f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.563653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0398633-cf32-4dfd-b5f8-b17cbdf7f9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.593399 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b0398633-cf32-4dfd-b5f8-b17cbdf7f9d8 None None] MFA Rules not processed for user `1063122c1462470fa74b40a6d43e0ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:15.594848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b0398633-cf32-4dfd-b5f8-b17cbdf7f9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:15.660757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1082/2147] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:06:15.665265 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea666e-4b44-4dff-aaa2-4162a0596aac admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:15.754690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea666e-4b44-4dff-aaa2-4162a0596aac admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:15.771295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ae51-c5df-44e1-9193-a6005d4d5b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:15.771939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ae51-c5df-44e1-9193-a6005d4d5b8c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.772221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ae51-c5df-44e1-9193-a6005d4d5b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.772496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5ae51-c5df-44e1-9193-a6005d4d5b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.773178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5ae51-c5df-44e1-9193-a6005d4d5b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:15.775124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5ae51-c5df-44e1-9193-a6005d4d5b8c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:15.787674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:15.828539 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1066/2148] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:06:15 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:15.832594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0398633-cf32-4dfd-b5f8-b17cbdf7f9d8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:15.864412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0398633-cf32-4dfd-b5f8-b17cbdf7f9d8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:15.919632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2b9ce-55cd-488b-8aae-e324be1c8b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:15.920943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2b9ce-55cd-488b-8aae-e324be1c8b41 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.921232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2b9ce-55cd-488b-8aae-e324be1c8b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.921511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2b9ce-55cd-488b-8aae-e324be1c8b41 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.922197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2b9ce-55cd-488b-8aae-e324be1c8b41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:15.924195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2b9ce-55cd-488b-8aae-e324be1c8b41 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:15.947826 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1083/2149] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:06:15 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:15.949185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ae51-c5df-44e1-9193-a6005d4d5b8c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:15.959478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5ae51-c5df-44e1-9193-a6005d4d5b8c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:15.964044 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5365b34-9db6-4588-a467-68167e4b2fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:15.964690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5365b34-9db6-4588-a467-68167e4b2fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.964970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5365b34-9db6-4588-a467-68167e4b2fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.965245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5365b34-9db6-4588-a467-68167e4b2fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:15.965963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5365b34-9db6-4588-a467-68167e4b2fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:15.968156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5365b34-9db6-4588-a467-68167e4b2fc8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:15.977982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1067/2150] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:15.982130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2b9ce-55cd-488b-8aae-e324be1c8b41 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:15.993372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2b9ce-55cd-488b-8aae-e324be1c8b41 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:15.998254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7df88-ab3e-4629-be81-300fbd2d6a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:15.999180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7df88-ab3e-4629-be81-300fbd2d6a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:15.999464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7df88-ab3e-4629-be81-300fbd2d6a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:15.999746 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7df88-ab3e-4629-be81-300fbd2d6a3b admin admin] PATH_INFO: `/v3/projects/589fc05d6f7443538eb2db60b970d9a3/users/82c3285920024cadb17a9e607f5d133f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:16.002881 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:16.016185 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7df88-ab3e-4629-be81-300fbd2d6a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=589fc05d6f7443538eb2db60b970d9a3, user_id=82c3285920024cadb17a9e607f5d133f, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:16.018906 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7df88-ab3e-4629-be81-300fbd2d6a3b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:16.032343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1084/2151] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:15 2022] PUT /identity/v3/projects/589fc05d6f7443538eb2db60b970d9a3/users/82c3285920024cadb17a9e607f5d133f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:16.036309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5365b34-9db6-4588-a467-68167e4b2fc8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:16.046801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5365b34-9db6-4588-a467-68167e4b2fc8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:16.101344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c99f40-1659-48d3-a20e-dbb6ec49563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:16.102135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c99f40-1659-48d3-a20e-dbb6ec49563a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:16.102513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c99f40-1659-48d3-a20e-dbb6ec49563a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:16.102805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c99f40-1659-48d3-a20e-dbb6ec49563a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:16.103525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c99f40-1659-48d3-a20e-dbb6ec49563a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:16.105136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c99f40-1659-48d3-a20e-dbb6ec49563a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:16.114987 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1068/2152] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:16.119126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7df88-ab3e-4629-be81-300fbd2d6a3b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:16.129841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7df88-ab3e-4629-be81-300fbd2d6a3b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:16.134386 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad177a-3f66-46e6-b94d-7e6fcd726093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:16.135293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad177a-3f66-46e6-b94d-7e6fcd726093 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:16.135577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad177a-3f66-46e6-b94d-7e6fcd726093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:16.135855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad177a-3f66-46e6-b94d-7e6fcd726093 admin admin] PATH_INFO: `/v3/projects/589fc05d6f7443538eb2db60b970d9a3/users/82c3285920024cadb17a9e607f5d133f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:16.137815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad177a-3f66-46e6-b94d-7e6fcd726093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=589fc05d6f7443538eb2db60b970d9a3, user_id=82c3285920024cadb17a9e607f5d133f, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:16.140969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad177a-3f66-46e6-b94d-7e6fcd726093 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:16.151454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1085/2153] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:16 2022] PUT /identity/v3/projects/589fc05d6f7443538eb2db60b970d9a3/users/82c3285920024cadb17a9e607f5d133f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:16.161972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e470c9-b034-418e-a3a8-566e9327386a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:16.162250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e470c9-b034-418e-a3a8-566e9327386a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:16.162564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e470c9-b034-418e-a3a8-566e9327386a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:16.189332 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c3e470c9-b034-418e-a3a8-566e9327386a None None] MFA Rules not processed for user `82c3285920024cadb17a9e607f5d133f`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:16.190568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c3e470c9-b034-418e-a3a8-566e9327386a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:16.279739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1069/2154] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:16.291792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad177a-3f66-46e6-b94d-7e6fcd726093 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:16.293545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ccad177a-3f66-46e6-b94d-7e6fcd726093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:16.371890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad177a-3f66-46e6-b94d-7e6fcd726093 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:16.376462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ac839-f6ea-47ed-955c-4dc7119525c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:16.376989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ac839-f6ea-47ed-955c-4dc7119525c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:16.377305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ac839-f6ea-47ed-955c-4dc7119525c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:16.377585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ac839-f6ea-47ed-955c-4dc7119525c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:16.386933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ac839-f6ea-47ed-955c-4dc7119525c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:16.388399 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ac839-f6ea-47ed-955c-4dc7119525c1 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:16.398028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1086/2155] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:16.725029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e470c9-b034-418e-a3a8-566e9327386a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:16.735377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e470c9-b034-418e-a3a8-566e9327386a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:16.740160 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8fccd-7f1c-433c-aa73-cbbe104ecf6e tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82c3285920024cadb17a9e607f5d133f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '589fc05d6f7443538eb2db60b970d9a3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:16.740823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8fccd-7f1c-433c-aa73-cbbe104ecf6e tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:16.741116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8fccd-7f1c-433c-aa73-cbbe104ecf6e tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:16.741736 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8fccd-7f1c-433c-aa73-cbbe104ecf6e tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:16.742513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8fccd-7f1c-433c-aa73-cbbe104ecf6e tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:16.742830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8fccd-7f1c-433c-aa73-cbbe104ecf6e tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:16.744467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf8fccd-7f1c-433c-aa73-cbbe104ecf6e tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:16.757012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1070/2156] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:16.761169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ac839-f6ea-47ed-955c-4dc7119525c1 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:16.772703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ac839-f6ea-47ed-955c-4dc7119525c1 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:16.777671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aa6e5-e607-4e3d-9e4d-47b8e845f659 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82c3285920024cadb17a9e607f5d133f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '589fc05d6f7443538eb2db60b970d9a3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:16.778333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aa6e5-e607-4e3d-9e4d-47b8e845f659 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:16.778659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aa6e5-e607-4e3d-9e4d-47b8e845f659 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:16.778944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aa6e5-e607-4e3d-9e4d-47b8e845f659 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:16.780156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29aa6e5-e607-4e3d-9e4d-47b8e845f659 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:16.782060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29aa6e5-e607-4e3d-9e4d-47b8e845f659 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:16.813044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1087/2157] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:06:16 2022] POST /identity/v3/projects => generated 352 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:16.817301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8fccd-7f1c-433c-aa73-cbbe104ecf6e tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:16.835430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf8fccd-7f1c-433c-aa73-cbbe104ecf6e tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:16.866779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:16.903470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d070efe-348c-4382-970f-9cf3193bede8 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82c3285920024cadb17a9e607f5d133f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '589fc05d6f7443538eb2db60b970d9a3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:16.904058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d070efe-348c-4382-970f-9cf3193bede8 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:16.904351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d070efe-348c-4382-970f-9cf3193bede8 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:16.905189 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d070efe-348c-4382-970f-9cf3193bede8 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:16.905928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d070efe-348c-4382-970f-9cf3193bede8 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:16.906258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d070efe-348c-4382-970f-9cf3193bede8 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:16.907940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d070efe-348c-4382-970f-9cf3193bede8 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:16.921993 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1071/2158] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:16.958139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aa6e5-e607-4e3d-9e4d-47b8e845f659 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:16.969302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aa6e5-e607-4e3d-9e4d-47b8e845f659 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:16.974268 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82c3285920024cadb17a9e607f5d133f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '589fc05d6f7443538eb2db60b970d9a3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:16.975173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:16.975461 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:16.975741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] PATH_INFO: `/v3/projects/59125393b925442797f2c76d4f9b8df2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:16.976958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: Authorizing `identity:delete_project(project_id=59125393b925442797f2c76d4f9b8df2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:16.978919 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:17.028868 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59125393b925442797f2c76d4f9b8df2', 'request_id': 'req-a2a35c12-2398-4e81-98b0-c9af919fdc75'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:17.036116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59125393b925442797f2c76d4f9b8df2', 'request_id': 'req-a2a35c12-2398-4e81-98b0-c9af919fdc75'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:17.041923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59125393b925442797f2c76d4f9b8df2', 'request_id': 'req-a2a35c12-2398-4e81-98b0-c9af919fdc75'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:17.056008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] The token cache is being invalidate because project 59125393b925442797f2c76d4f9b8df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:17.056466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:17.057113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a2a35c12-2398-4e81-98b0-c9af919fdc75 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:17.058742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1088/2159] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:16 2022] DELETE /identity/v3/projects/59125393b925442797f2c76d4f9b8df2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:17.247609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d070efe-348c-4382-970f-9cf3193bede8 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:17.249792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9d070efe-348c-4382-970f-9cf3193bede8 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:17.314820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d070efe-348c-4382-970f-9cf3193bede8 tempest-VolumeQuotaClassesTest-936872203 tempest-VolumeQuotaClassesTest-936872203-project-admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:17.369759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:17.370651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:17.370933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:17.371208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] PATH_INFO: `/v3/users/1063122c1462470fa74b40a6d43e0ac4` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:17.372646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1063122c1462470fa74b40a6d43e0ac4)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:17.375143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:17.419600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1063122c1462470fa74b40a6d43e0ac4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:17.420193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1063122c1462470fa74b40a6d43e0ac4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:17.427034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1063122c1462470fa74b40a6d43e0ac4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:17.427655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1063122c1462470fa74b40a6d43e0ac4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:17.439990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1063122c1462470fa74b40a6d43e0ac4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:17.441933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:17.454363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1072/2160] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:17 2022] DELETE /identity/v3/users/1063122c1462470fa74b40a6d43e0ac4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:17.458923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd616f99-cbb5-4a4b-9f61-f4931c776c96 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:17.459404 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd616f99-cbb5-4a4b-9f61-f4931c776c96 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:17.459518 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd616f99-cbb5-4a4b-9f61-f4931c776c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:17.486427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bd616f99-cbb5-4a4b-9f61-f4931c776c96 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:17.487385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bd616f99-cbb5-4a4b-9f61-f4931c776c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:17.496629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:17.564751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1089/2161] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:17.574606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:17.576104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:17.661099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98711c32-b8b1-466e-9f51-afdf9e2d106b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:17.665173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223af81-c80d-4c50-b370-b9bad9f470c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:17.665879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223af81-c80d-4c50-b370-b9bad9f470c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:17.666162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223af81-c80d-4c50-b370-b9bad9f470c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:17.666454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223af81-c80d-4c50-b370-b9bad9f470c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:17.676629 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223af81-c80d-4c50-b370-b9bad9f470c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:17.678168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223af81-c80d-4c50-b370-b9bad9f470c4 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:17.688756 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1073/2162] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:17.945682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd616f99-cbb5-4a4b-9f61-f4931c776c96 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:17.947155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bd616f99-cbb5-4a4b-9f61-f4931c776c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:17.959390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd616f99-cbb5-4a4b-9f61-f4931c776c96 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:17.964903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:17.965788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:17.966137 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:17.966331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] PATH_INFO: `/v3/projects/881780ecfb4d4b0f871f9e79bed72e3b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:17.968058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=881780ecfb4d4b0f871f9e79bed72e3b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:17.970791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:18.019536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '881780ecfb4d4b0f871f9e79bed72e3b', 'request_id': 'req-5210c057-7b38-48f0-be4a-8d26aa1e93d7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.030198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '881780ecfb4d4b0f871f9e79bed72e3b', 'request_id': 'req-5210c057-7b38-48f0-be4a-8d26aa1e93d7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.035833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '881780ecfb4d4b0f871f9e79bed72e3b', 'request_id': 'req-5210c057-7b38-48f0-be4a-8d26aa1e93d7'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.048812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] The token cache is being invalidate because project 881780ecfb4d4b0f871f9e79bed72e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:18.049101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.049707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.051295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1090/2163] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:17 2022] DELETE /identity/v3/projects/881780ecfb4d4b0f871f9e79bed72e3b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:18.055321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223af81-c80d-4c50-b370-b9bad9f470c4 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:18.064853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223af81-c80d-4c50-b370-b9bad9f470c4 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:18.119904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:18.120758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:18.121037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:18.121307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] PATH_INFO: `/v3/users/82c3285920024cadb17a9e607f5d133f` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:18.122908 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c3285920024cadb17a9e607f5d133f)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:18.124868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:18.171176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c3285920024cadb17a9e607f5d133f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.171810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c3285920024cadb17a9e607f5d133f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.178674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c3285920024cadb17a9e607f5d133f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.179278 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c3285920024cadb17a9e607f5d133f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.190914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3d0bef11-d81b-44cc-978b-abb9b7e5537d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c3285920024cadb17a9e607f5d133f'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.195340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:18.205337 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1074/2164] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:18 2022] DELETE /identity/v3/users/82c3285920024cadb17a9e607f5d133f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:18.478798 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:18.480423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:18.494922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210c057-7b38-48f0-be4a-8d26aa1e93d7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:18.551651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:18.552499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:18.552780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:18.553053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] PATH_INFO: `/v3/projects/589fc05d6f7443538eb2db60b970d9a3` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:18.554217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=589fc05d6f7443538eb2db60b970d9a3)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:18.556192 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:18.597300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:18.605288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '589fc05d6f7443538eb2db60b970d9a3', 'request_id': 'req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.611553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '589fc05d6f7443538eb2db60b970d9a3', 'request_id': 'req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.617028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '589fc05d6f7443538eb2db60b970d9a3', 'request_id': 'req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.628650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] The token cache is being invalidate because project 589fc05d6f7443538eb2db60b970d9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:18.628938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.629560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3dbcf8c5-77af-4127-b99c-b0b57fa9f102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:18.631147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1091/2165] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:18 2022] DELETE /identity/v3/projects/589fc05d6f7443538eb2db60b970d9a3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:18.635972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7cd6e-bb92-4f1c-896e-dae1c8754968 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:18.636455 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7cd6e-bb92-4f1c-896e-dae1c8754968 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:18.636546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7cd6e-bb92-4f1c-896e-dae1c8754968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:18.675013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-25e7cd6e-bb92-4f1c-896e-dae1c8754968 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:18.676238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-25e7cd6e-bb92-4f1c-896e-dae1c8754968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:18.743564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1075/2166] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:18.745520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e67f3-e778-4958-a559-b3d795f8cd58 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:18.745852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e67f3-e778-4958-a559-b3d795f8cd58 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:18.746067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e67f3-e778-4958-a559-b3d795f8cd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:18.791790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2d7e67f3-e778-4958-a559-b3d795f8cd58 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:18.793037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2d7e67f3-e778-4958-a559-b3d795f8cd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:18.802398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1092/2167] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:18.806868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7cd6e-bb92-4f1c-896e-dae1c8754968 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:18.824382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7cd6e-bb92-4f1c-896e-dae1c8754968 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:18.830130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a860cf2-e12f-4316-ac14-2eda527ddbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:18.831060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a860cf2-e12f-4316-ac14-2eda527ddbde admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:18.831525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a860cf2-e12f-4316-ac14-2eda527ddbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:18.832167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a860cf2-e12f-4316-ac14-2eda527ddbde admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:18.833208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a860cf2-e12f-4316-ac14-2eda527ddbde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:18.833665 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a860cf2-e12f-4316-ac14-2eda527ddbde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:18.835458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a860cf2-e12f-4316-ac14-2eda527ddbde admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:18.849529 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1076/2168] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:18.854058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e67f3-e778-4958-a559-b3d795f8cd58 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:18.863698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e67f3-e778-4958-a559-b3d795f8cd58 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:18.868894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942add21-89f0-4bff-9a09-d7d8d683c9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:18.869233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942add21-89f0-4bff-9a09-d7d8d683c9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:18.869584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942add21-89f0-4bff-9a09-d7d8d683c9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:18.869929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942add21-89f0-4bff-9a09-d7d8d683c9f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:18.870749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942add21-89f0-4bff-9a09-d7d8d683c9f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:18.872936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942add21-89f0-4bff-9a09-d7d8d683c9f1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:18.901315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1093/2169] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:06:18 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:18.905572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a860cf2-e12f-4316-ac14-2eda527ddbde admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:18.915709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a860cf2-e12f-4316-ac14-2eda527ddbde admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:18.975640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b1b5c-6d78-442f-ad1c-190de944cf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:18.976385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b1b5c-6d78-442f-ad1c-190de944cf68 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:18.976677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b1b5c-6d78-442f-ad1c-190de944cf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:18.976958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b1b5c-6d78-442f-ad1c-190de944cf68 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:18.977661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b1b5c-6d78-442f-ad1c-190de944cf68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:18.979637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b1b5c-6d78-442f-ad1c-190de944cf68 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.002758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1077/2170] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:06:18 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:06:19.006880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942add21-89f0-4bff-9a09-d7d8d683c9f1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.017705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942add21-89f0-4bff-9a09-d7d8d683c9f1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.022262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaead25d-9064-4d44-b2c1-b253b81e8b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.022964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaead25d-9064-4d44-b2c1-b253b81e8b40 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.023249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaead25d-9064-4d44-b2c1-b253b81e8b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.023530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaead25d-9064-4d44-b2c1-b253b81e8b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.024245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaead25d-9064-4d44-b2c1-b253b81e8b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.025849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaead25d-9064-4d44-b2c1-b253b81e8b40 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.036205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1094/2171] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:06:19.040153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b1b5c-6d78-442f-ad1c-190de944cf68 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.050019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b1b5c-6d78-442f-ad1c-190de944cf68 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.054611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e458c17-413d-41b2-8ced-386272bc994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.055266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e458c17-413d-41b2-8ced-386272bc994c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.055551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e458c17-413d-41b2-8ced-386272bc994c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.055849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e458c17-413d-41b2-8ced-386272bc994c admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.056549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e458c17-413d-41b2-8ced-386272bc994c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.058158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e458c17-413d-41b2-8ced-386272bc994c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.068346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1078/2172] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:06:19.072298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaead25d-9064-4d44-b2c1-b253b81e8b40 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.082722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaead25d-9064-4d44-b2c1-b253b81e8b40 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.087328 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfe1d8-e8e2-40ca-a390-b31d66f3af1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.088194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfe1d8-e8e2-40ca-a390-b31d66f3af1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.088480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfe1d8-e8e2-40ca-a390-b31d66f3af1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.088759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfe1d8-e8e2-40ca-a390-b31d66f3af1f admin admin] PATH_INFO: `/v3/projects/9dc437357d8a431dacbd642ed2f689d2/users/dd046829c71d41469d0a0955c3461dbf/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.103213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfe1d8-e8e2-40ca-a390-b31d66f3af1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc437357d8a431dacbd642ed2f689d2, user_id=dd046829c71d41469d0a0955c3461dbf, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.105856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfe1d8-e8e2-40ca-a390-b31d66f3af1f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.118964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1095/2173] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:19 2022] PUT /identity/v3/projects/9dc437357d8a431dacbd642ed2f689d2/users/dd046829c71d41469d0a0955c3461dbf/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:19.129783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b0300-3a8e-4d5e-a991-300f931a2b7f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.130077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b0300-3a8e-4d5e-a991-300f931a2b7f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.130356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b0300-3a8e-4d5e-a991-300f931a2b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.156048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-453b0300-3a8e-4d5e-a991-300f931a2b7f None None] MFA Rules not processed for user `dd046829c71d41469d0a0955c3461dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:19.157245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-453b0300-3a8e-4d5e-a991-300f931a2b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:19.189464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:19.221845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1079/2174] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:19.226292 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfe1d8-e8e2-40ca-a390-b31d66f3af1f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.286916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfe1d8-e8e2-40ca-a390-b31d66f3af1f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.291557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d2734-c107-4497-824e-f249c2708afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.292206 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d2734-c107-4497-824e-f249c2708afc admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.292491 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d2734-c107-4497-824e-f249c2708afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.292771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d2734-c107-4497-824e-f249c2708afc admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.293465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d2734-c107-4497-824e-f249c2708afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.295935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d2734-c107-4497-824e-f249c2708afc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.320599 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1096/2175] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:06:19 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:06:19.324745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b0300-3a8e-4d5e-a991-300f931a2b7f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.335210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b0300-3a8e-4d5e-a991-300f931a2b7f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.392494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723ef3a-936d-496d-b248-7e734c1e475f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.393166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723ef3a-936d-496d-b248-7e734c1e475f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.393451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723ef3a-936d-496d-b248-7e734c1e475f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.393731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723ef3a-936d-496d-b248-7e734c1e475f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.394425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723ef3a-936d-496d-b248-7e734c1e475f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.396401 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723ef3a-936d-496d-b248-7e734c1e475f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.420530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1080/2176] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:06:19 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:19.421604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d2734-c107-4497-824e-f249c2708afc admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.431930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d2734-c107-4497-824e-f249c2708afc admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.437312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463d5bf-99ea-4f73-b42c-490ee0072f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.437969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463d5bf-99ea-4f73-b42c-490ee0072f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.438260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463d5bf-99ea-4f73-b42c-490ee0072f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.438571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463d5bf-99ea-4f73-b42c-490ee0072f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.439302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463d5bf-99ea-4f73-b42c-490ee0072f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.440902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463d5bf-99ea-4f73-b42c-490ee0072f93 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.451357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1097/2177] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:06:19.455376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723ef3a-936d-496d-b248-7e734c1e475f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.465187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723ef3a-936d-496d-b248-7e734c1e475f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.470290 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ed4bd-128c-4019-8959-a7de9368a979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.471193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ed4bd-128c-4019-8959-a7de9368a979 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.471479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ed4bd-128c-4019-8959-a7de9368a979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.471755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ed4bd-128c-4019-8959-a7de9368a979 admin admin] PATH_INFO: `/v3/projects/85031e765fa54dbaae359650c5dee060/users/5d1be257dcd447bb9b053419b8711528/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.486770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ed4bd-128c-4019-8959-a7de9368a979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85031e765fa54dbaae359650c5dee060, user_id=5d1be257dcd447bb9b053419b8711528, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.489392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ed4bd-128c-4019-8959-a7de9368a979 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.502216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1081/2178] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:19 2022] PUT /identity/v3/projects/85031e765fa54dbaae359650c5dee060/users/5d1be257dcd447bb9b053419b8711528/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:19.506057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463d5bf-99ea-4f73-b42c-490ee0072f93 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.515964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463d5bf-99ea-4f73-b42c-490ee0072f93 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.549009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:19.573740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8958b091-c9aa-435c-97d3-04219c23fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.574401 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958b091-c9aa-435c-97d3-04219c23fbad admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.574722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958b091-c9aa-435c-97d3-04219c23fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.575005 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8958b091-c9aa-435c-97d3-04219c23fbad admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.575727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958b091-c9aa-435c-97d3-04219c23fbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.577354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8958b091-c9aa-435c-97d3-04219c23fbad admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.586957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1098/2179] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:19.590986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ed4bd-128c-4019-8959-a7de9368a979 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.601558 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ed4bd-128c-4019-8959-a7de9368a979 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.606570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d5154-09a7-4be9-a7cc-b5ec6e3d93c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.607530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d5154-09a7-4be9-a7cc-b5ec6e3d93c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.608357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d5154-09a7-4be9-a7cc-b5ec6e3d93c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.608653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3d5154-09a7-4be9-a7cc-b5ec6e3d93c9 admin admin] PATH_INFO: `/v3/projects/85031e765fa54dbaae359650c5dee060/users/5d1be257dcd447bb9b053419b8711528/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.610642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d5154-09a7-4be9-a7cc-b5ec6e3d93c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85031e765fa54dbaae359650c5dee060, user_id=5d1be257dcd447bb9b053419b8711528, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.613355 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3d5154-09a7-4be9-a7cc-b5ec6e3d93c9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.623539 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1082/2180] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:19 2022] PUT /identity/v3/projects/85031e765fa54dbaae359650c5dee060/users/5d1be257dcd447bb9b053419b8711528/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:19.633328 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51967f-e5f9-47a5-a861-f39517b387a0 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.633621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51967f-e5f9-47a5-a861-f39517b387a0 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.633905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f51967f-e5f9-47a5-a861-f39517b387a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.662348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2f51967f-e5f9-47a5-a861-f39517b387a0 None None] MFA Rules not processed for user `5d1be257dcd447bb9b053419b8711528`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:19.663583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2f51967f-e5f9-47a5-a861-f39517b387a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:19.729933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1099/2181] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:19.741262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d5154-09a7-4be9-a7cc-b5ec6e3d93c9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.743032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cb3d5154-09a7-4be9-a7cc-b5ec6e3d93c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:19.813009 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3d5154-09a7-4be9-a7cc-b5ec6e3d93c9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.817448 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbead79-6049-4c4a-ab08-c5b482a1faf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.818112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbead79-6049-4c4a-ab08-c5b482a1faf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.818424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbead79-6049-4c4a-ab08-c5b482a1faf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.818748 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbead79-6049-4c4a-ab08-c5b482a1faf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.828628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbead79-6049-4c4a-ab08-c5b482a1faf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.830074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbead79-6049-4c4a-ab08-c5b482a1faf4 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.840460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1083/2182] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:20.263542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51967f-e5f9-47a5-a861-f39517b387a0 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:20.327764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f51967f-e5f9-47a5-a861-f39517b387a0 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:20.332390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:20.333257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:20.333541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:20.333814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] PATH_INFO: `/v3/users/dd046829c71d41469d0a0955c3461dbf` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:20.335082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd046829c71d41469d0a0955c3461dbf)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:20.337130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:20.383879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd046829c71d41469d0a0955c3461dbf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:20.395439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:20.398052 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd046829c71d41469d0a0955c3461dbf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:20.406207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd046829c71d41469d0a0955c3461dbf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:20.406859 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd046829c71d41469d0a0955c3461dbf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:20.413739 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd046829c71d41469d0a0955c3461dbf'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:20.419730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1100/2183] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:20 2022] DELETE /identity/v3/users/dd046829c71d41469d0a0955c3461dbf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:20.425914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11962f64-2a22-45eb-a497-62aab880d78f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:20.425914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11962f64-2a22-45eb-a497-62aab880d78f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:20.425914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11962f64-2a22-45eb-a497-62aab880d78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:20.447094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-11962f64-2a22-45eb-a497-62aab880d78f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:20.448474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-11962f64-2a22-45eb-a497-62aab880d78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:20.475366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:20.545546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1084/2184] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:20.558229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:20.569121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:20.619669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a8dcb-c610-4ce8-898e-14b10bcdc7f8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:20.624205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848a25a-f80e-48d6-884d-54fa02211aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:20.624854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848a25a-f80e-48d6-884d-54fa02211aee service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:20.625140 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848a25a-f80e-48d6-884d-54fa02211aee service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:20.625468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848a25a-f80e-48d6-884d-54fa02211aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:20.634656 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848a25a-f80e-48d6-884d-54fa02211aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:20.636114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848a25a-f80e-48d6-884d-54fa02211aee service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:20.646232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1101/2185] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:20.940832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11962f64-2a22-45eb-a497-62aab880d78f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:20.942461 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-11962f64-2a22-45eb-a497-62aab880d78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:20.956958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11962f64-2a22-45eb-a497-62aab880d78f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:20.963511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:20.964898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:20.965451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:20.965808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] PATH_INFO: `/v3/projects/9dc437357d8a431dacbd642ed2f689d2` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:20.971097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc437357d8a431dacbd642ed2f689d2)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:20.971097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:21.024899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc437357d8a431dacbd642ed2f689d2', 'request_id': 'req-35dabff4-58a2-4428-a33e-ac034ad2b94c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.031254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc437357d8a431dacbd642ed2f689d2', 'request_id': 'req-35dabff4-58a2-4428-a33e-ac034ad2b94c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.037834 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc437357d8a431dacbd642ed2f689d2', 'request_id': 'req-35dabff4-58a2-4428-a33e-ac034ad2b94c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.051779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] The token cache is being invalidate because project 9dc437357d8a431dacbd642ed2f689d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:21.052064 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.052685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.054317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1085/2186] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:20 2022] DELETE /identity/v3/projects/9dc437357d8a431dacbd642ed2f689d2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:21.058299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848a25a-f80e-48d6-884d-54fa02211aee service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:21.077657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848a25a-f80e-48d6-884d-54fa02211aee service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:21.138965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:21.139836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.140121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.140402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] PATH_INFO: `/v3/users/5d1be257dcd447bb9b053419b8711528` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.141585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1be257dcd447bb9b053419b8711528)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:21.143558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:21.189148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1be257dcd447bb9b053419b8711528'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.200885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1be257dcd447bb9b053419b8711528'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.207880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1be257dcd447bb9b053419b8711528'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.208585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1be257dcd447bb9b053419b8711528'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.215636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-397cee15-7428-4e64-bb57-802b13fd358d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1be257dcd447bb9b053419b8711528'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.218256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:21.226574 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1102/2187] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:21 2022] DELETE /identity/v3/users/5d1be257dcd447bb9b053419b8711528 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:21.497078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:21.498592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:21.511477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dabff4-58a2-4428-a33e-ac034ad2b94c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:21.567854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:21.568708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.568992 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.569266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] PATH_INFO: `/v3/projects/85031e765fa54dbaae359650c5dee060` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.570436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85031e765fa54dbaae359650c5dee060)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:21.572440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:21.617625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85031e765fa54dbaae359650c5dee060', 'request_id': 'req-da753ebd-91cf-423a-981c-e83ab5a34606'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.621511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:21.623940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85031e765fa54dbaae359650c5dee060', 'request_id': 'req-da753ebd-91cf-423a-981c-e83ab5a34606'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.632262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85031e765fa54dbaae359650c5dee060', 'request_id': 'req-da753ebd-91cf-423a-981c-e83ab5a34606'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.645039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] The token cache is being invalidate because project 85031e765fa54dbaae359650c5dee060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:21.645461 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.646424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-da753ebd-91cf-423a-981c-e83ab5a34606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:21.648299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1086/2188] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:21 2022] DELETE /identity/v3/projects/85031e765fa54dbaae359650c5dee060 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:21.653091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc08223-5bdc-416f-9d4c-d7b77b53a4c6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.653538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc08223-5bdc-416f-9d4c-d7b77b53a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.653643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc08223-5bdc-416f-9d4c-d7b77b53a4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.671495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8bc08223-5bdc-416f-9d4c-d7b77b53a4c6 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:21.672695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8bc08223-5bdc-416f-9d4c-d7b77b53a4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:21.730018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1103/2189] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:21.737429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00f703-8a81-4060-bf5b-30692bc4d7a2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.737722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00f703-8a81-4060-bf5b-30692bc4d7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.738029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00f703-8a81-4060-bf5b-30692bc4d7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.756654 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8b00f703-8a81-4060-bf5b-30692bc4d7a2 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:21.757831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8b00f703-8a81-4060-bf5b-30692bc4d7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:21.766179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1087/2190] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:21.770077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc08223-5bdc-416f-9d4c-d7b77b53a4c6 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:21.779542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc08223-5bdc-416f-9d4c-d7b77b53a4c6 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:21.784856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfaca31-593d-4c0a-8420-426c8570a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:21.785510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaca31-593d-4c0a-8420-426c8570a4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.785793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaca31-593d-4c0a-8420-426c8570a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.786070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaca31-593d-4c0a-8420-426c8570a4d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.786861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfaca31-593d-4c0a-8420-426c8570a4d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:21.787170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfaca31-593d-4c0a-8420-426c8570a4d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:21.788759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfaca31-593d-4c0a-8420-426c8570a4d6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:21.800121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1104/2191] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:21.804271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00f703-8a81-4060-bf5b-30692bc4d7a2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:21.814203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00f703-8a81-4060-bf5b-30692bc4d7a2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:21.818859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63de721-cee6-456a-9337-5ab0080197f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:21.819540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63de721-cee6-456a-9337-5ab0080197f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.819822 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63de721-cee6-456a-9337-5ab0080197f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.820121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63de721-cee6-456a-9337-5ab0080197f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.820810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63de721-cee6-456a-9337-5ab0080197f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:21.822963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63de721-cee6-456a-9337-5ab0080197f2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:21.849051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1088/2192] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:06:21 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:21.852979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfaca31-593d-4c0a-8420-426c8570a4d6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:21.862849 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfaca31-593d-4c0a-8420-426c8570a4d6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:21.919705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489a2a9-77ae-43a7-9421-5af5faecbaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:21.920416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489a2a9-77ae-43a7-9421-5af5faecbaea admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.920703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489a2a9-77ae-43a7-9421-5af5faecbaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.921004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489a2a9-77ae-43a7-9421-5af5faecbaea admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.921685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489a2a9-77ae-43a7-9421-5af5faecbaea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:21.923633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489a2a9-77ae-43a7-9421-5af5faecbaea admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:21.946994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1105/2193] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:06:21 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:21.948171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63de721-cee6-456a-9337-5ab0080197f2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:21.959265 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63de721-cee6-456a-9337-5ab0080197f2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:21.964040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da475b92-0dcf-439e-b180-2a2bf6e517f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:21.964695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da475b92-0dcf-439e-b180-2a2bf6e517f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.964979 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da475b92-0dcf-439e-b180-2a2bf6e517f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.965257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da475b92-0dcf-439e-b180-2a2bf6e517f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.965981 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da475b92-0dcf-439e-b180-2a2bf6e517f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:21.967675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da475b92-0dcf-439e-b180-2a2bf6e517f5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:21.978145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1089/2194] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:21.982243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489a2a9-77ae-43a7-9421-5af5faecbaea admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:21.992194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489a2a9-77ae-43a7-9421-5af5faecbaea admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:21.997687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83823e6-1035-4f96-9fff-e870fd4321e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:21.998907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83823e6-1035-4f96-9fff-e870fd4321e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.999192 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83823e6-1035-4f96-9fff-e870fd4321e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.999474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83823e6-1035-4f96-9fff-e870fd4321e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:22.000294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83823e6-1035-4f96-9fff-e870fd4321e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:22.001805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83823e6-1035-4f96-9fff-e870fd4321e8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:22.012418 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1106/2195] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:22.017493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da475b92-0dcf-439e-b180-2a2bf6e517f5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:22.027820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da475b92-0dcf-439e-b180-2a2bf6e517f5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:22.032342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a892f8-199f-4dbe-bdf1-efa84dfdcf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:22.033252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a892f8-199f-4dbe-bdf1-efa84dfdcf8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:22.033607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a892f8-199f-4dbe-bdf1-efa84dfdcf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:22.033887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a892f8-199f-4dbe-bdf1-efa84dfdcf8f admin admin] PATH_INFO: `/v3/projects/8fd02dca48af44859e371cf04a8e87d4/users/1ee210dd40434fd0bded0365f536377d/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:22.048467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a892f8-199f-4dbe-bdf1-efa84dfdcf8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd02dca48af44859e371cf04a8e87d4, user_id=1ee210dd40434fd0bded0365f536377d, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:22.051163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a892f8-199f-4dbe-bdf1-efa84dfdcf8f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:22.064435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1090/2196] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:22 2022] PUT /identity/v3/projects/8fd02dca48af44859e371cf04a8e87d4/users/1ee210dd40434fd0bded0365f536377d/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:22.069455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95750b3d-bbe9-4400-b7d5-03668d2f346f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:22.069753 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95750b3d-bbe9-4400-b7d5-03668d2f346f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:22.070036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95750b3d-bbe9-4400-b7d5-03668d2f346f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:22.088068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-95750b3d-bbe9-4400-b7d5-03668d2f346f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:22.089205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-95750b3d-bbe9-4400-b7d5-03668d2f346f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:22.148268 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1107/2197] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:22.157679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a892f8-199f-4dbe-bdf1-efa84dfdcf8f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:22.159389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-70a892f8-199f-4dbe-bdf1-efa84dfdcf8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:22.217535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a892f8-199f-4dbe-bdf1-efa84dfdcf8f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:22.221244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3667-5d90-4527-8b2e-7dd8ee5c652b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:22.221943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3667-5d90-4527-8b2e-7dd8ee5c652b service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:22.222226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3667-5d90-4527-8b2e-7dd8ee5c652b service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:22.222538 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3667-5d90-4527-8b2e-7dd8ee5c652b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:22.232165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f3667-5d90-4527-8b2e-7dd8ee5c652b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:22.234144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f3667-5d90-4527-8b2e-7dd8ee5c652b service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:22.244308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1091/2198] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:23.120287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaa2ec-b2fd-4ee0-9166-c5af9d0beeb2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:23.120818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaa2ec-b2fd-4ee0-9166-c5af9d0beeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:23.120935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaa2ec-b2fd-4ee0-9166-c5af9d0beeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:23.136296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:23.153434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6cbaa2ec-b2fd-4ee0-9166-c5af9d0beeb2 None None] MFA Rules not processed for user `1ee210dd40434fd0bded0365f536377d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:23.155402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6cbaa2ec-b2fd-4ee0-9166-c5af9d0beeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:23.230930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1108/2199] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:06:23.236036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3667-5d90-4527-8b2e-7dd8ee5c652b service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:23.248693 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3667-5d90-4527-8b2e-7dd8ee5c652b service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:23.253360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778d747-ebf1-4da1-b967-859a1f4d911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:23.253934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778d747-ebf1-4da1-b967-859a1f4d911e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:23.254214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778d747-ebf1-4da1-b967-859a1f4d911e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:23.254516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778d747-ebf1-4da1-b967-859a1f4d911e admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:23.255216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778d747-ebf1-4da1-b967-859a1f4d911e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:23.257117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778d747-ebf1-4da1-b967-859a1f4d911e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:23.289156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1092/2200] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:06:23 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:23.290693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaa2ec-b2fd-4ee0-9166-c5af9d0beeb2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:23.302655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaa2ec-b2fd-4ee0-9166-c5af9d0beeb2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:23.365956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83228d-8862-44a1-a74a-e1c938883f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:23.366684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83228d-8862-44a1-a74a-e1c938883f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:23.366980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83228d-8862-44a1-a74a-e1c938883f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:23.367265 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83228d-8862-44a1-a74a-e1c938883f33 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:23.367987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83228d-8862-44a1-a74a-e1c938883f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:23.369967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83228d-8862-44a1-a74a-e1c938883f33 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:23.393534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1109/2201] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:06:23 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:23.397601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778d747-ebf1-4da1-b967-859a1f4d911e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:23.408984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778d747-ebf1-4da1-b967-859a1f4d911e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:23.413807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae83319-cb29-4e30-bd70-e3ec5a78e2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:23.414527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae83319-cb29-4e30-bd70-e3ec5a78e2eb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:23.414818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae83319-cb29-4e30-bd70-e3ec5a78e2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:23.415102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae83319-cb29-4e30-bd70-e3ec5a78e2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:23.415839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae83319-cb29-4e30-bd70-e3ec5a78e2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:23.417490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae83319-cb29-4e30-bd70-e3ec5a78e2eb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:23.430141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1093/2202] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:23.434363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83228d-8862-44a1-a74a-e1c938883f33 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:23.444349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83228d-8862-44a1-a74a-e1c938883f33 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:23.448891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e444a3c-362f-44f2-84af-d368d47b00b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:23.449769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e444a3c-362f-44f2-84af-d368d47b00b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:23.450058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e444a3c-362f-44f2-84af-d368d47b00b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:23.450341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e444a3c-362f-44f2-84af-d368d47b00b5 admin admin] PATH_INFO: `/v3/projects/89303b66d68b4a31a2daffcb65be783c/users/0da2e54cb7d445cea1da2030d61d3412/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:23.465132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e444a3c-362f-44f2-84af-d368d47b00b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89303b66d68b4a31a2daffcb65be783c, user_id=0da2e54cb7d445cea1da2030d61d3412, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:23.468375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e444a3c-362f-44f2-84af-d368d47b00b5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:23.480815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1110/2203] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:23 2022] PUT /identity/v3/projects/89303b66d68b4a31a2daffcb65be783c/users/0da2e54cb7d445cea1da2030d61d3412/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:23.485430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae83319-cb29-4e30-bd70-e3ec5a78e2eb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:23.495862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae83319-cb29-4e30-bd70-e3ec5a78e2eb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:23.543103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:23.556844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27400812-b9ad-4eb5-883c-11437ae23b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:23.557507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27400812-b9ad-4eb5-883c-11437ae23b49 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:23.557790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27400812-b9ad-4eb5-883c-11437ae23b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:23.558067 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27400812-b9ad-4eb5-883c-11437ae23b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:23.558819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27400812-b9ad-4eb5-883c-11437ae23b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:23.560445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27400812-b9ad-4eb5-883c-11437ae23b49 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:23.571312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1094/2204] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:23.575096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e444a3c-362f-44f2-84af-d368d47b00b5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:23.589082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e444a3c-362f-44f2-84af-d368d47b00b5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:23.595257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836eb6e-322b-4969-b6ff-310b09fa5bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:23.596983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836eb6e-322b-4969-b6ff-310b09fa5bf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:23.597400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836eb6e-322b-4969-b6ff-310b09fa5bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:23.597686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836eb6e-322b-4969-b6ff-310b09fa5bf6 admin admin] PATH_INFO: `/v3/projects/89303b66d68b4a31a2daffcb65be783c/users/0da2e54cb7d445cea1da2030d61d3412/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:23.599715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836eb6e-322b-4969-b6ff-310b09fa5bf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89303b66d68b4a31a2daffcb65be783c, user_id=0da2e54cb7d445cea1da2030d61d3412, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:23.602785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836eb6e-322b-4969-b6ff-310b09fa5bf6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:23.616029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1111/2205] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:23 2022] PUT /identity/v3/projects/89303b66d68b4a31a2daffcb65be783c/users/0da2e54cb7d445cea1da2030d61d3412/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:24.484368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae8ef6-afff-447f-954e-e3f8c0ed513a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.484845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae8ef6-afff-447f-954e-e3f8c0ed513a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.484928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae8ef6-afff-447f-954e-e3f8c0ed513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.514211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-caae8ef6-afff-447f-954e-e3f8c0ed513a None None] MFA Rules not processed for user `0da2e54cb7d445cea1da2030d61d3412`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:24.515591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-caae8ef6-afff-447f-954e-e3f8c0ed513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:24.594330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1095/2206] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:24.605424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836eb6e-322b-4969-b6ff-310b09fa5bf6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:24.607956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e836eb6e-322b-4969-b6ff-310b09fa5bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:24.666346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:24.671455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836eb6e-322b-4969-b6ff-310b09fa5bf6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:24.675908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c2632-1172-4198-8fe6-5c4d8d63a4f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:24.676557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c2632-1172-4198-8fe6-5c4d8d63a4f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.676844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c2632-1172-4198-8fe6-5c4d8d63a4f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.677121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c2632-1172-4198-8fe6-5c4d8d63a4f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.734526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c2632-1172-4198-8fe6-5c4d8d63a4f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:24.735968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c2632-1172-4198-8fe6-5c4d8d63a4f6 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:24.745159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1112/2207] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:26.161676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae8ef6-afff-447f-954e-e3f8c0ed513a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:26.172299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae8ef6-afff-447f-954e-e3f8c0ed513a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:26.176077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef2a15-6930-4b2f-98db-7e948fe97d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:26.176934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef2a15-6930-4b2f-98db-7e948fe97d78 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:26.177226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef2a15-6930-4b2f-98db-7e948fe97d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:26.177554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef2a15-6930-4b2f-98db-7e948fe97d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:26.191002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef2a15-6930-4b2f-98db-7e948fe97d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:26.192702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef2a15-6930-4b2f-98db-7e948fe97d78 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:26.204037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1096/2208] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:31.117337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fbe7e-9824-404b-99be-6abc14620106 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:31.117337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fbe7e-9824-404b-99be-6abc14620106 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:31.117337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fbe7e-9824-404b-99be-6abc14620106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:31.141067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-053fbe7e-9824-404b-99be-6abc14620106 None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:31.142245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-053fbe7e-9824-404b-99be-6abc14620106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:31.195597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1113/2209] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:31.235148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef2a15-6930-4b2f-98db-7e948fe97d78 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:31.236545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-92ef2a15-6930-4b2f-98db-7e948fe97d78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:31.313687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef2a15-6930-4b2f-98db-7e948fe97d78 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:31.320729 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5754a3-7c23-4b53-8f3e-926db98ae220 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:31.321296 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5754a3-7c23-4b53-8f3e-926db98ae220 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:31.321582 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5754a3-7c23-4b53-8f3e-926db98ae220 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:31.321863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5754a3-7c23-4b53-8f3e-926db98ae220 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:31.332552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5754a3-7c23-4b53-8f3e-926db98ae220 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:31.334197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5754a3-7c23-4b53-8f3e-926db98ae220 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:31.343398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1097/2210] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:06:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:06:41.188911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=55 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:41.339305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=79 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:41.855539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a3ac1-4a8e-4ff3-9d95-c2d9b1e36cd8 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:41.856105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a3ac1-4a8e-4ff3-9d95-c2d9b1e36cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:41.856543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a3ac1-4a8e-4ff3-9d95-c2d9b1e36cd8 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:41.858924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1114/2211] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:41.864675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec26cf-3581-4acf-b365-3584a8f76140 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:41.865161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec26cf-3581-4acf-b365-3584a8f76140 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:41.865594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec26cf-3581-4acf-b365-3584a8f76140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:41.942806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ebec26cf-3581-4acf-b365-3584a8f76140 None None] MFA Rules not processed for user `4edb53dc871047ce89ebb3b90143a6de`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:41.944145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ebec26cf-3581-4acf-b365-3584a8f76140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:42.038609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1098/2212] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:06:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:06:42.057977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a3ac1-4a8e-4ff3-9d95-c2d9b1e36cd8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:42.059963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-af9a3ac1-4a8e-4ff3-9d95-c2d9b1e36cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:42.078566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a3ac1-4a8e-4ff3-9d95-c2d9b1e36cd8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:42.085214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744c017-c736-4676-90df-310c0ac3137d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:42.086140 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744c017-c736-4676-90df-310c0ac3137d service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:42.086650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744c017-c736-4676-90df-310c0ac3137d service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:42.087134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744c017-c736-4676-90df-310c0ac3137d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:42.100623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744c017-c736-4676-90df-310c0ac3137d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:42.102646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744c017-c736-4676-90df-310c0ac3137d service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:42.126539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1115/2213] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:42.846436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec26cf-3581-4acf-b365-3584a8f76140 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:42.916952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec26cf-3581-4acf-b365-3584a8f76140 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:42.922630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:42.923510 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:42.923796 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:42.924073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] PATH_INFO: `/v3/users/1ee210dd40434fd0bded0365f536377d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:42.925267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ee210dd40434fd0bded0365f536377d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:42.927549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:43.012694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ee210dd40434fd0bded0365f536377d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.013791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ee210dd40434fd0bded0365f536377d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.027912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ee210dd40434fd0bded0365f536377d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.029485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ee210dd40434fd0bded0365f536377d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.033289 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:43.050716 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ee210dd40434fd0bded0365f536377d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.098074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1099/2214] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:42 2022] DELETE /identity/v3/users/1ee210dd40434fd0bded0365f536377d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:43.354543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744c017-c736-4676-90df-310c0ac3137d service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:43.373221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744c017-c736-4676-90df-310c0ac3137d service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:43.472205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:43.473937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:43.474475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:43.474985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] PATH_INFO: `/v3/projects/8fd02dca48af44859e371cf04a8e87d4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:43.476448 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd02dca48af44859e371cf04a8e87d4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:43.478659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:43.553894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd02dca48af44859e371cf04a8e87d4', 'request_id': 'req-1e106f2e-3983-400d-9db9-185ef0d726b9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.562195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd02dca48af44859e371cf04a8e87d4', 'request_id': 'req-1e106f2e-3983-400d-9db9-185ef0d726b9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.569851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd02dca48af44859e371cf04a8e87d4', 'request_id': 'req-1e106f2e-3983-400d-9db9-185ef0d726b9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.613011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] The token cache is being invalidate because project 8fd02dca48af44859e371cf04a8e87d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:43.613225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.613920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.618748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1116/2215] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:43 2022] DELETE /identity/v3/projects/8fd02dca48af44859e371cf04a8e87d4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:43.622736 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:43.625918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:43.661415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe3b43-038d-43ef-85c3-a0cd63a5186b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:43.769138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:43.770305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:43.770744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:43.771155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] PATH_INFO: `/v3/users/0da2e54cb7d445cea1da2030d61d3412` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:43.772515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da2e54cb7d445cea1da2030d61d3412)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:43.774511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:43.850424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da2e54cb7d445cea1da2030d61d3412'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.851754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da2e54cb7d445cea1da2030d61d3412'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.866316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da2e54cb7d445cea1da2030d61d3412'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.867324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da2e54cb7d445cea1da2030d61d3412'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.890028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6f997309-98b7-44f0-9779-ff6467795246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da2e54cb7d445cea1da2030d61d3412'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:43.914679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1100/2216] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:43 2022] DELETE /identity/v3/users/0da2e54cb7d445cea1da2030d61d3412 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:44.156763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.158421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:44.178018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e106f2e-3983-400d-9db9-185ef0d726b9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:44.268534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:44.270257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.270907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.274338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] PATH_INFO: `/v3/projects/89303b66d68b4a31a2daffcb65be783c` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.277445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89303b66d68b4a31a2daffcb65be783c)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:44.279814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:44.305006 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:44.348572 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89303b66d68b4a31a2daffcb65be783c', 'request_id': 'req-04fe7997-e077-4965-ac19-2e3e07436198'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:44.370584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89303b66d68b4a31a2daffcb65be783c', 'request_id': 'req-04fe7997-e077-4965-ac19-2e3e07436198'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:44.377908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89303b66d68b4a31a2daffcb65be783c', 'request_id': 'req-04fe7997-e077-4965-ac19-2e3e07436198'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:44.398014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] The token cache is being invalidate because project 89303b66d68b4a31a2daffcb65be783c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:44.398304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:44.399061 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-04fe7997-e077-4965-ac19-2e3e07436198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:44.403926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1117/2217] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:44 2022] DELETE /identity/v3/projects/89303b66d68b4a31a2daffcb65be783c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:44.405820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b757e3-0a3e-4b42-9e0f-94be94f63988 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.406114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b757e3-0a3e-4b42-9e0f-94be94f63988 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.406402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b757e3-0a3e-4b42-9e0f-94be94f63988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.433902 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-85b757e3-0a3e-4b42-9e0f-94be94f63988 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:44.435238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-85b757e3-0a3e-4b42-9e0f-94be94f63988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:44.522060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:44.546917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1101/2218] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:44.555245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8618-8379-471d-88e3-399fa6b966aa None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.555536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8618-8379-471d-88e3-399fa6b966aa None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.555818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e8618-8379-471d-88e3-399fa6b966aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.576750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8e0e8618-8379-471d-88e3-399fa6b966aa None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:44.578028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8e0e8618-8379-471d-88e3-399fa6b966aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:44.586622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1118/2219] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:44.590909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b757e3-0a3e-4b42-9e0f-94be94f63988 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.601269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b757e3-0a3e-4b42-9e0f-94be94f63988 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:44.607113 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1c281-0558-48bf-bea8-340b7feea9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:44.608325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1c281-0558-48bf-bea8-340b7feea9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.608959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1c281-0558-48bf-bea8-340b7feea9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.609244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1c281-0558-48bf-bea8-340b7feea9c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.609995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1c281-0558-48bf-bea8-340b7feea9c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:44.610304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1c281-0558-48bf-bea8-340b7feea9c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:44.612002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1c281-0558-48bf-bea8-340b7feea9c8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:44.633594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1102/2220] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:44.634838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e8618-8379-471d-88e3-399fa6b966aa None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.644731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e8618-8379-471d-88e3-399fa6b966aa None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:44.650941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe893c-169e-411f-abd6-d52481f5c5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:44.651553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe893c-169e-411f-abd6-d52481f5c5df admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.651841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe893c-169e-411f-abd6-d52481f5c5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.652121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe893c-169e-411f-abd6-d52481f5c5df admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.652827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe893c-169e-411f-abd6-d52481f5c5df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:44.654911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe893c-169e-411f-abd6-d52481f5c5df admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:44.685069 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1119/2221] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:06:44 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:44.696044 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1c281-0558-48bf-bea8-340b7feea9c8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.711363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1c281-0558-48bf-bea8-340b7feea9c8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:44.775346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4457f0e-a01e-401f-bf6c-9f9a17b8dd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:44.776014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4457f0e-a01e-401f-bf6c-9f9a17b8dd5f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.776300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4457f0e-a01e-401f-bf6c-9f9a17b8dd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.776742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4457f0e-a01e-401f-bf6c-9f9a17b8dd5f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.777454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4457f0e-a01e-401f-bf6c-9f9a17b8dd5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:44.779597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4457f0e-a01e-401f-bf6c-9f9a17b8dd5f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:44.822324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1103/2222] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:06:44 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:06:44.826053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe893c-169e-411f-abd6-d52481f5c5df admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.838925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe893c-169e-411f-abd6-d52481f5c5df admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:44.845166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660a84c-e274-47a2-9223-85366acc5d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:44.846655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660a84c-e274-47a2-9223-85366acc5d23 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.846937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660a84c-e274-47a2-9223-85366acc5d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.847232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660a84c-e274-47a2-9223-85366acc5d23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.847964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660a84c-e274-47a2-9223-85366acc5d23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:44.850568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660a84c-e274-47a2-9223-85366acc5d23 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:44.866735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1120/2223] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:06:44.870310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4457f0e-a01e-401f-bf6c-9f9a17b8dd5f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.883495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4457f0e-a01e-401f-bf6c-9f9a17b8dd5f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:44.890415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b0fa7-c4d2-4f59-8b1d-a146f5ed5535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:44.891632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b0fa7-c4d2-4f59-8b1d-a146f5ed5535 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.892266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b0fa7-c4d2-4f59-8b1d-a146f5ed5535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.893425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b0fa7-c4d2-4f59-8b1d-a146f5ed5535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.894390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b0fa7-c4d2-4f59-8b1d-a146f5ed5535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:44.896310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b0fa7-c4d2-4f59-8b1d-a146f5ed5535 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:44.914178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1104/2224] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:44.915016 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660a84c-e274-47a2-9223-85366acc5d23 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.936345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660a84c-e274-47a2-9223-85366acc5d23 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:44.948912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e2bb2-a30f-4216-a386-9c14cbb68023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:44.949903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2bb2-a30f-4216-a386-9c14cbb68023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.950261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2bb2-a30f-4216-a386-9c14cbb68023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.950651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2bb2-a30f-4216-a386-9c14cbb68023 admin admin] PATH_INFO: `/v3/projects/8be014681c31482c94ce5ba41173a5ce/users/c0c656fd818545fa84da69149e409d0e/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.968474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e2bb2-a30f-4216-a386-9c14cbb68023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8be014681c31482c94ce5ba41173a5ce, user_id=c0c656fd818545fa84da69149e409d0e, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:44.971464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e2bb2-a30f-4216-a386-9c14cbb68023 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:44.990729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1121/2225] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:44 2022] PUT /identity/v3/projects/8be014681c31482c94ce5ba41173a5ce/users/c0c656fd818545fa84da69149e409d0e/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:44.999624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574feac-0b46-49f2-9898-f1cbb3fe4b46 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.999991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574feac-0b46-49f2-9898-f1cbb3fe4b46 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.000341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574feac-0b46-49f2-9898-f1cbb3fe4b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.029370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c574feac-0b46-49f2-9898-f1cbb3fe4b46 None None] MFA Rules not processed for user `c0c656fd818545fa84da69149e409d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:45.030646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c574feac-0b46-49f2-9898-f1cbb3fe4b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:45.106170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1105/2226] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:45.117130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e2bb2-a30f-4216-a386-9c14cbb68023 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:45.118870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-970e2bb2-a30f-4216-a386-9c14cbb68023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:45.188322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e2bb2-a30f-4216-a386-9c14cbb68023 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.192120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651428f-6973-4329-a21b-0c28c8fb3859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.192776 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651428f-6973-4329-a21b-0c28c8fb3859 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.193067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651428f-6973-4329-a21b-0c28c8fb3859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.193350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651428f-6973-4329-a21b-0c28c8fb3859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.206124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651428f-6973-4329-a21b-0c28c8fb3859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:45.207899 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651428f-6973-4329-a21b-0c28c8fb3859 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:45.217861 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1122/2227] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:06:48.626477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574feac-0b46-49f2-9898-f1cbb3fe4b46 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:48.682355 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:48.699314 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574feac-0b46-49f2-9898-f1cbb3fe4b46 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:48.704528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:48.705630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:48.706160 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:48.706462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] PATH_INFO: `/v3/users/c0c656fd818545fa84da69149e409d0e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:48.707701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0c656fd818545fa84da69149e409d0e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:48.709701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:48.769765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0c656fd818545fa84da69149e409d0e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:48.770695 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0c656fd818545fa84da69149e409d0e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:48.780527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0c656fd818545fa84da69149e409d0e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:48.781508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0c656fd818545fa84da69149e409d0e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:48.801112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0c656fd818545fa84da69149e409d0e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:48.833976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1106/2228] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:48 2022] DELETE /identity/v3/users/c0c656fd818545fa84da69149e409d0e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:48.838903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2278b-7297-443b-8b3b-46f432d33836 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:48.839409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2278b-7297-443b-8b3b-46f432d33836 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:48.839614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2278b-7297-443b-8b3b-46f432d33836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:48.867253 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-0bf2278b-7297-443b-8b3b-46f432d33836 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:48.868467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0bf2278b-7297-443b-8b3b-46f432d33836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:48.923542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:48.953417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1123/2229] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:06:48.964536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:48.965512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:49.052324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256c34e-a5bc-4b61-a0c3-ddea2c05455d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:49.056448 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99d26b7-173c-4f13-a744-341fddd6a59f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:49.057322 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d26b7-173c-4f13-a744-341fddd6a59f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:49.057676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d26b7-173c-4f13-a744-341fddd6a59f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:49.058049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99d26b7-173c-4f13-a744-341fddd6a59f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:49.075275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d26b7-173c-4f13-a744-341fddd6a59f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:49.077586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99d26b7-173c-4f13-a744-341fddd6a59f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:49.088775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1107/2230] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:49.527158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2278b-7297-443b-8b3b-46f432d33836 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:49.528714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-0bf2278b-7297-443b-8b3b-46f432d33836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:49.546659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2278b-7297-443b-8b3b-46f432d33836 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:49.553281 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:49.554398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:49.554754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:49.554984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] PATH_INFO: `/v3/projects/8be014681c31482c94ce5ba41173a5ce` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:49.556357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8be014681c31482c94ce5ba41173a5ce)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:49.558282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:49.611325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8be014681c31482c94ce5ba41173a5ce', 'request_id': 'req-e1d09aa3-9464-4816-8a36-beac6c39bce3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:49.618920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8be014681c31482c94ce5ba41173a5ce', 'request_id': 'req-e1d09aa3-9464-4816-8a36-beac6c39bce3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:49.627226 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8be014681c31482c94ce5ba41173a5ce', 'request_id': 'req-e1d09aa3-9464-4816-8a36-beac6c39bce3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:49.645332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] The token cache is being invalidate because project 8be014681c31482c94ce5ba41173a5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:49.645698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:49.646616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:49.648622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1124/2231] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:49 2022] DELETE /identity/v3/projects/8be014681c31482c94ce5ba41173a5ce => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:49.982149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfcb9f-1da8-4c36-80f5-e3364f9d25b0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:49.983343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfcb9f-1da8-4c36-80f5-e3364f9d25b0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:49.983438 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfcb9f-1da8-4c36-80f5-e3364f9d25b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.001921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a7dfcb9f-1da8-4c36-80f5-e3364f9d25b0 None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:50.003149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a7dfcb9f-1da8-4c36-80f5-e3364f9d25b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:50.068165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1108/2232] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:50.080946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:50.082462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:50.158232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d09aa3-9464-4816-8a36-beac6c39bce3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:50.163191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb39cc-01c0-425a-907e-6d3b10279276 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:50.163864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb39cc-01c0-425a-907e-6d3b10279276 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.164154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb39cc-01c0-425a-907e-6d3b10279276 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.164435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb39cc-01c0-425a-907e-6d3b10279276 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.173580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb39cc-01c0-425a-907e-6d3b10279276 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:50.175349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb39cc-01c0-425a-907e-6d3b10279276 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:50.184305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1125/2233] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:06:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:06:50.623895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe85a0-b3cb-49da-9fc9-fd2e2d78594c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.624187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe85a0-b3cb-49da-9fc9-fd2e2d78594c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.624468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe85a0-b3cb-49da-9fc9-fd2e2d78594c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.664463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1ebe85a0-b3cb-49da-9fc9-fd2e2d78594c None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:50.665884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1ebe85a0-b3cb-49da-9fc9-fd2e2d78594c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:50.687543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1109/2234] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 26 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:50.699693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e8470-32ba-46d0-bc61-0d4303cc46f5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.700625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e8470-32ba-46d0-bc61-0d4303cc46f5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.700928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e8470-32ba-46d0-bc61-0d4303cc46f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.706272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe85a0-b3cb-49da-9fc9-fd2e2d78594c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:50.716686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:50.724873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-370e8470-32ba-46d0-bc61-0d4303cc46f5 None None] MFA Rules not processed for user `443be2bc27514432a34d2a4c57108b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:50.726238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-370e8470-32ba-46d0-bc61-0d4303cc46f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:50.726384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe85a0-b3cb-49da-9fc9-fd2e2d78594c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:50.740710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972842b-0bd3-4c63-ae5d-73b5cec2c8fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:50.741590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972842b-0bd3-4c63-ae5d-73b5cec2c8fb service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.742183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972842b-0bd3-4c63-ae5d-73b5cec2c8fb service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.742306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972842b-0bd3-4c63-ae5d-73b5cec2c8fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.772887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972842b-0bd3-4c63-ae5d-73b5cec2c8fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:50.774685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972842b-0bd3-4c63-ae5d-73b5cec2c8fb service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:50.793166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1110/2235] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:06:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 21:06:50.850962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1126/2236] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:53.077476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972842b-0bd3-4c63-ae5d-73b5cec2c8fb service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:53.079136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4972842b-0bd3-4c63-ae5d-73b5cec2c8fb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:53.218299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972842b-0bd3-4c63-ae5d-73b5cec2c8fb service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:53.229660 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:53.230657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.230965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.231229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] PATH_INFO: `/v3/users/4a3f5eeb2f01498aa4c84c5ecc6a6cbf` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.233018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a3f5eeb2f01498aa4c84c5ecc6a6cbf)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:53.235042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:53.330088 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a3f5eeb2f01498aa4c84c5ecc6a6cbf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:53.331066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a3f5eeb2f01498aa4c84c5ecc6a6cbf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:53.344385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a3f5eeb2f01498aa4c84c5ecc6a6cbf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:53.345434 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a3f5eeb2f01498aa4c84c5ecc6a6cbf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:53.363499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-0cfdb014-a13a-4525-a9fa-aeefe1cb6d81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a3f5eeb2f01498aa4c84c5ecc6a6cbf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:53.381805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1111/2237] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:53 2022] DELETE /identity/v3/users/4a3f5eeb2f01498aa4c84c5ecc6a6cbf => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:53.626956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e8470-32ba-46d0-bc61-0d4303cc46f5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:53.641133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:53.646810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e8470-32ba-46d0-bc61-0d4303cc46f5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:53.736116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:53.737187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.737597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.738111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] PATH_INFO: `/v3/projects/9a6a63e6c17d4b8bbd698f6e652de935` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.741439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a6a63e6c17d4b8bbd698f6e652de935)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:53.741701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:53.815316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a6a63e6c17d4b8bbd698f6e652de935', 'request_id': 'req-57745fd7-68d1-40d2-8ad9-77113aa525b3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:53.823955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a6a63e6c17d4b8bbd698f6e652de935', 'request_id': 'req-57745fd7-68d1-40d2-8ad9-77113aa525b3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:53.831092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a6a63e6c17d4b8bbd698f6e652de935', 'request_id': 'req-57745fd7-68d1-40d2-8ad9-77113aa525b3'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:53.859343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] The token cache is being invalidate because project 9a6a63e6c17d4b8bbd698f6e652de935 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:53.859537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:53.860255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-57745fd7-68d1-40d2-8ad9-77113aa525b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:53.862021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1127/2238] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:53 2022] DELETE /identity/v3/projects/9a6a63e6c17d4b8bbd698f6e652de935 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:53.869467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8000a5-cddc-4398-9819-913d6f020fa7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.869760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8000a5-cddc-4398-9819-913d6f020fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.870050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8000a5-cddc-4398-9819-913d6f020fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.892935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-eb8000a5-cddc-4398-9819-913d6f020fa7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:53.894386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eb8000a5-cddc-4398-9819-913d6f020fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:53.970766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1112/2239] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:53.974009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8016cc-3906-4273-8363-120239107f82 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.974307 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8016cc-3906-4273-8363-120239107f82 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.974618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8016cc-3906-4273-8363-120239107f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.006276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6d8016cc-3906-4273-8363-120239107f82 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:54.008048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6d8016cc-3906-4273-8363-120239107f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:54.023000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1128/2240] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:54.024005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8000a5-cddc-4398-9819-913d6f020fa7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.034166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8000a5-cddc-4398-9819-913d6f020fa7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.040891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85f3f5-f641-4dca-a339-b4543a487744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.041462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85f3f5-f641-4dca-a339-b4543a487744 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.041752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85f3f5-f641-4dca-a339-b4543a487744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.042036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85f3f5-f641-4dca-a339-b4543a487744 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.042811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85f3f5-f641-4dca-a339-b4543a487744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:54.043122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85f3f5-f641-4dca-a339-b4543a487744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.044804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85f3f5-f641-4dca-a339-b4543a487744 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.048662 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:54.065540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1113/2241] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:54.066632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8016cc-3906-4273-8363-120239107f82 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.078223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8016cc-3906-4273-8363-120239107f82 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.088687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e25f56-96f8-4cde-abda-49cc4f61e200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.089375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e25f56-96f8-4cde-abda-49cc4f61e200 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.089663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e25f56-96f8-4cde-abda-49cc4f61e200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.089938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e25f56-96f8-4cde-abda-49cc4f61e200 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.090683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e25f56-96f8-4cde-abda-49cc4f61e200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.092674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e25f56-96f8-4cde-abda-49cc4f61e200 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.136728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1129/2242] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:06:54 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:54.141388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85f3f5-f641-4dca-a339-b4543a487744 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.153205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85f3f5-f641-4dca-a339-b4543a487744 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.243001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a74e63-ed78-404f-9db3-af80bb66b7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.243819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a74e63-ed78-404f-9db3-af80bb66b7c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.244114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a74e63-ed78-404f-9db3-af80bb66b7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.244395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a74e63-ed78-404f-9db3-af80bb66b7c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.245284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a74e63-ed78-404f-9db3-af80bb66b7c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.247340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a74e63-ed78-404f-9db3-af80bb66b7c7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.298754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1114/2243] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:06:54 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:54.299759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e25f56-96f8-4cde-abda-49cc4f61e200 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.318396 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e25f56-96f8-4cde-abda-49cc4f61e200 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.331114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bf6d8-72cd-4be6-a2a0-8ab16236ed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.331805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bf6d8-72cd-4be6-a2a0-8ab16236ed93 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.332091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bf6d8-72cd-4be6-a2a0-8ab16236ed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.332375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bf6d8-72cd-4be6-a2a0-8ab16236ed93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.333108 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bf6d8-72cd-4be6-a2a0-8ab16236ed93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.334850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bf6d8-72cd-4be6-a2a0-8ab16236ed93 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.366657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1130/2244] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:06:54.367929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a74e63-ed78-404f-9db3-af80bb66b7c7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.380552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a74e63-ed78-404f-9db3-af80bb66b7c7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.397300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b918c34-5e91-4c15-8758-bb0210871b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.398090 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b918c34-5e91-4c15-8758-bb0210871b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.402623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b918c34-5e91-4c15-8758-bb0210871b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.405989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b918c34-5e91-4c15-8758-bb0210871b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.407047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b918c34-5e91-4c15-8758-bb0210871b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.408813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b918c34-5e91-4c15-8758-bb0210871b1a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.428047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1115/2245] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:54.431900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bf6d8-72cd-4be6-a2a0-8ab16236ed93 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.456754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bf6d8-72cd-4be6-a2a0-8ab16236ed93 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.466932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a433706-a0e6-4d5b-a962-73c598109650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.467998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a433706-a0e6-4d5b-a962-73c598109650 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.468528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a433706-a0e6-4d5b-a962-73c598109650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.468912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a433706-a0e6-4d5b-a962-73c598109650 admin admin] PATH_INFO: `/v3/projects/afa629d403944fb09023592b59a3debb/users/b7f74242db5441888d481399ebb9b927/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.509855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a433706-a0e6-4d5b-a962-73c598109650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa629d403944fb09023592b59a3debb, user_id=b7f74242db5441888d481399ebb9b927, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.512513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a433706-a0e6-4d5b-a962-73c598109650 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.548917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1131/2246] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:54 2022] PUT /identity/v3/projects/afa629d403944fb09023592b59a3debb/users/b7f74242db5441888d481399ebb9b927/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:54.572333 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd4f26-8d8f-477c-8e92-1c20c3ff9468 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.572855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd4f26-8d8f-477c-8e92-1c20c3ff9468 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.573221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd4f26-8d8f-477c-8e92-1c20c3ff9468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.652283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0cfd4f26-8d8f-477c-8e92-1c20c3ff9468 None None] MFA Rules not processed for user `b7f74242db5441888d481399ebb9b927`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:54.654056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0cfd4f26-8d8f-477c-8e92-1c20c3ff9468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:54.740608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1116/2247] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:54.753700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a433706-a0e6-4d5b-a962-73c598109650 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.755766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4a433706-a0e6-4d5b-a962-73c598109650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:54.803573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:54.870328 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a433706-a0e6-4d5b-a962-73c598109650 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.889510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af2a30-6d0e-45e9-a61a-534819d70419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.890210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af2a30-6d0e-45e9-a61a-534819d70419 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.890525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af2a30-6d0e-45e9-a61a-534819d70419 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.890811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af2a30-6d0e-45e9-a61a-534819d70419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.900370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af2a30-6d0e-45e9-a61a-534819d70419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.901771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af2a30-6d0e-45e9-a61a-534819d70419 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.915086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1132/2248] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:57.624728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd4f26-8d8f-477c-8e92-1c20c3ff9468 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:57.716041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd4f26-8d8f-477c-8e92-1c20c3ff9468 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:57.723515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:57.724433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:57.724848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:57.725168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] PATH_INFO: `/v3/users/b7f74242db5441888d481399ebb9b927` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:57.726805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7f74242db5441888d481399ebb9b927)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:57.728942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:57.734480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:57.818998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7f74242db5441888d481399ebb9b927'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:57.820552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7f74242db5441888d481399ebb9b927'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:57.832145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7f74242db5441888d481399ebb9b927'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:57.833150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7f74242db5441888d481399ebb9b927'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:57.851664 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7f74242db5441888d481399ebb9b927'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:57.867820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1117/2249] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:06:57 2022] DELETE /identity/v3/users/b7f74242db5441888d481399ebb9b927 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:57.874325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d8f11-931b-4af1-ad66-1e5a06daffc7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:57.874851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d8f11-931b-4af1-ad66-1e5a06daffc7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:57.874976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d8f11-931b-4af1-ad66-1e5a06daffc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:57.896964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f85d8f11-931b-4af1-ad66-1e5a06daffc7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:57.899087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f85d8f11-931b-4af1-ad66-1e5a06daffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:57.979183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1133/2250] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:57.989853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:57.990981 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:58.059137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cafae9-0a07-44be-8bbb-1807dd848377 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:58.064194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da575fef-affb-401e-828f-bfb114183dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:58.064854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da575fef-affb-401e-828f-bfb114183dcb service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.065137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da575fef-affb-401e-828f-bfb114183dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.065445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da575fef-affb-401e-828f-bfb114183dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.075514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da575fef-affb-401e-828f-bfb114183dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:58.077005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da575fef-affb-401e-828f-bfb114183dcb service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:58.087405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1118/2251] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:06:58.487577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d8f11-931b-4af1-ad66-1e5a06daffc7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:58.489124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f85d8f11-931b-4af1-ad66-1e5a06daffc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:58.509277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d8f11-931b-4af1-ad66-1e5a06daffc7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:58.514139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:58.515119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.515477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.515821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] PATH_INFO: `/v3/projects/afa629d403944fb09023592b59a3debb` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.517116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa629d403944fb09023592b59a3debb)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:58.519165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:58.588561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa629d403944fb09023592b59a3debb', 'request_id': 'req-55716f8c-e16b-4307-b34d-e22057d2097d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:58.596235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa629d403944fb09023592b59a3debb', 'request_id': 'req-55716f8c-e16b-4307-b34d-e22057d2097d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:58.602604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa629d403944fb09023592b59a3debb', 'request_id': 'req-55716f8c-e16b-4307-b34d-e22057d2097d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:58.624616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] The token cache is being invalidate because project afa629d403944fb09023592b59a3debb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:06:58.624900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:58.625559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-55716f8c-e16b-4307-b34d-e22057d2097d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:06:58.627495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1134/2252] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:06:58 2022] DELETE /identity/v3/projects/afa629d403944fb09023592b59a3debb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:06:58.632294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d23c80-bc39-4373-aad0-7bc2a0f9e15c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.632763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d23c80-bc39-4373-aad0-7bc2a0f9e15c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.632882 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d23c80-bc39-4373-aad0-7bc2a0f9e15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.655717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-78d23c80-bc39-4373-aad0-7bc2a0f9e15c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:58.657445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78d23c80-bc39-4373-aad0-7bc2a0f9e15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:58.734338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1119/2253] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:58.742175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62910b63-2810-4c0e-936b-10a8c6bee7a3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.742467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62910b63-2810-4c0e-936b-10a8c6bee7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.742877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62910b63-2810-4c0e-936b-10a8c6bee7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.768365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-62910b63-2810-4c0e-936b-10a8c6bee7a3 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:58.769688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-62910b63-2810-4c0e-936b-10a8c6bee7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:58.782502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1135/2254] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:58.784956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d23c80-bc39-4373-aad0-7bc2a0f9e15c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:58.796790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d23c80-bc39-4373-aad0-7bc2a0f9e15c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:58.806274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da98031-b65e-4691-a55b-2d38254b11bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:58.806995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da98031-b65e-4691-a55b-2d38254b11bd admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.807288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da98031-b65e-4691-a55b-2d38254b11bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.807574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da98031-b65e-4691-a55b-2d38254b11bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.808514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da98031-b65e-4691-a55b-2d38254b11bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:58.808649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da98031-b65e-4691-a55b-2d38254b11bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:58.810318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da98031-b65e-4691-a55b-2d38254b11bd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:58.841008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1120/2255] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:58.845130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62910b63-2810-4c0e-936b-10a8c6bee7a3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:58.854504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62910b63-2810-4c0e-936b-10a8c6bee7a3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:58.866558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b121f-1dff-465c-8c2b-4345a15d4993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:58.867243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b121f-1dff-465c-8c2b-4345a15d4993 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.867529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b121f-1dff-465c-8c2b-4345a15d4993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.867813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b121f-1dff-465c-8c2b-4345a15d4993 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.868504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b121f-1dff-465c-8c2b-4345a15d4993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:58.870565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b121f-1dff-465c-8c2b-4345a15d4993 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:58.902946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1136/2256] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:06:58 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:58.906960 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da98031-b65e-4691-a55b-2d38254b11bd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:58.921164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da98031-b65e-4691-a55b-2d38254b11bd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:58.933062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:59.000309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a705576-78dd-43a1-8249-35f6c57e1a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.002026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a705576-78dd-43a1-8249-35f6c57e1a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.002370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a705576-78dd-43a1-8249-35f6c57e1a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.002746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a705576-78dd-43a1-8249-35f6c57e1a0c admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.003526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a705576-78dd-43a1-8249-35f6c57e1a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.005652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a705576-78dd-43a1-8249-35f6c57e1a0c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.035819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1121/2257] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:06:58 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:59.039717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b121f-1dff-465c-8c2b-4345a15d4993 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.057290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b121f-1dff-465c-8c2b-4345a15d4993 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.063822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16fff4-55ef-485c-81bb-d16f2e8c91ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.064645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16fff4-55ef-485c-81bb-d16f2e8c91ac admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.065013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16fff4-55ef-485c-81bb-d16f2e8c91ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.065393 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16fff4-55ef-485c-81bb-d16f2e8c91ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.066276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16fff4-55ef-485c-81bb-d16f2e8c91ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.067982 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16fff4-55ef-485c-81bb-d16f2e8c91ac admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.072714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:59.087544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1137/2258] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:59.092830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a705576-78dd-43a1-8249-35f6c57e1a0c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.104527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a705576-78dd-43a1-8249-35f6c57e1a0c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.115728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3eb57-aa46-4026-a5be-89a76049b416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.116416 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3eb57-aa46-4026-a5be-89a76049b416 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.116698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3eb57-aa46-4026-a5be-89a76049b416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.116974 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3eb57-aa46-4026-a5be-89a76049b416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.117707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3eb57-aa46-4026-a5be-89a76049b416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.119383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3eb57-aa46-4026-a5be-89a76049b416 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.133183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1122/2259] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:06:59.137018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16fff4-55ef-485c-81bb-d16f2e8c91ac admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.149914 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16fff4-55ef-485c-81bb-d16f2e8c91ac admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.155321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e0732-d25c-4895-b1b3-c96c4baed158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.156208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0732-d25c-4895-b1b3-c96c4baed158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.156490 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0732-d25c-4895-b1b3-c96c4baed158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.156762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0732-d25c-4895-b1b3-c96c4baed158 admin admin] PATH_INFO: `/v3/projects/35fbec6984b044b889e81fbba81ce8d6/users/ce5b08d2b78b482380db0502873c1e23/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.178077 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0732-d25c-4895-b1b3-c96c4baed158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35fbec6984b044b889e81fbba81ce8d6, user_id=ce5b08d2b78b482380db0502873c1e23, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.182959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0732-d25c-4895-b1b3-c96c4baed158 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.207020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1138/2260] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:06:59 2022] PUT /identity/v3/projects/35fbec6984b044b889e81fbba81ce8d6/users/ce5b08d2b78b482380db0502873c1e23/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:59.208812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8334dbd-eb40-429b-8135-af3a8c650ef8 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.209105 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8334dbd-eb40-429b-8135-af3a8c650ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.209383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8334dbd-eb40-429b-8135-af3a8c650ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.239391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e8334dbd-eb40-429b-8135-af3a8c650ef8 None None] MFA Rules not processed for user `ce5b08d2b78b482380db0502873c1e23`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:59.240653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e8334dbd-eb40-429b-8135-af3a8c650ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:59.323819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1123/2261] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:06:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:06:59.335986 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e0732-d25c-4895-b1b3-c96c4baed158 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.337828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-099e0732-d25c-4895-b1b3-c96c4baed158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:59.401305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e0732-d25c-4895-b1b3-c96c4baed158 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.409563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217d593-0a30-4a70-b197-a59d7cd6f95a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3bd8dd462846bf9423f8528aa5c489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.409563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d593-0a30-4a70-b197-a59d7cd6f95a service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.409563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d593-0a30-4a70-b197-a59d7cd6f95a service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.409563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d593-0a30-4a70-b197-a59d7cd6f95a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.420874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217d593-0a30-4a70-b197-a59d7cd6f95a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.422616 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217d593-0a30-4a70-b197-a59d7cd6f95a service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.433507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1139/2262] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:06:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:07:06.543362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2250b-2324-4ea1-8c6c-4222e7f9f0fc None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.544358 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2250b-2324-4ea1-8c6c-4222e7f9f0fc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.544809 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2250b-2324-4ea1-8c6c-4222e7f9f0fc None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.547279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1124/2263] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:07:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:06.555011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f4936-632a-48e0-afa4-2ce31e353c94 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.556095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f4936-632a-48e0-afa4-2ce31e353c94 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.556578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2f4936-632a-48e0-afa4-2ce31e353c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.621319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7e2f4936-632a-48e0-afa4-2ce31e353c94 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:06.622611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7e2f4936-632a-48e0-afa4-2ce31e353c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:06.715570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1140/2264] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:06.728116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2250b-2324-4ea1-8c6c-4222e7f9f0fc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:06.752891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2250b-2324-4ea1-8c6c-4222e7f9f0fc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:06.756739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78e404-c851-44d3-acef-cf591a801111 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3bd8dd462846bf9423f8528aa5c489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:06.757383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78e404-c851-44d3-acef-cf591a801111 service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.757668 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78e404-c851-44d3-acef-cf591a801111 service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.758109 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78e404-c851-44d3-acef-cf591a801111 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.770122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78e404-c851-44d3-acef-cf591a801111 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:06.771641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78e404-c851-44d3-acef-cf591a801111 service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:06.785792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1125/2265] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:07:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:07:06.866418 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521420d-ad39-4066-81f3-38e1ea72e4fc None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.866744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521420d-ad39-4066-81f3-38e1ea72e4fc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.867099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521420d-ad39-4066-81f3-38e1ea72e4fc None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.868494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1141/2266] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:06.874137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7239c9a-b10e-4147-ad8d-8ac9e67c6f11 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.874459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7239c9a-b10e-4147-ad8d-8ac9e67c6f11 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.874757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7239c9a-b10e-4147-ad8d-8ac9e67c6f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.896968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d7239c9a-b10e-4147-ad8d-8ac9e67c6f11 None None] MFA Rules not processed for user `94b8d8b089264b5192a651b41bf2a205`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:06.898165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d7239c9a-b10e-4147-ad8d-8ac9e67c6f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:06.952956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1126/2267] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:06.964916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521420d-ad39-4066-81f3-38e1ea72e4fc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:06.966685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6521420d-ad39-4066-81f3-38e1ea72e4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:06.980443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521420d-ad39-4066-81f3-38e1ea72e4fc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:06.984670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd1529-4b06-41d1-a413-ca3dcf1de8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:06.985323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1529-4b06-41d1-a413-ca3dcf1de8dc service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.985609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1529-4b06-41d1-a413-ca3dcf1de8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.985887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1529-4b06-41d1-a413-ca3dcf1de8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.994032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd1529-4b06-41d1-a413-ca3dcf1de8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:06.996572 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd1529-4b06-41d1-a413-ca3dcf1de8dc service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:07.007376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1142/2268] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:07:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:07:07.147072 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205f3b9-610f-43e3-98a4-2c0cd69b6b5f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.147386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205f3b9-610f-43e3-98a4-2c0cd69b6b5f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.147641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205f3b9-610f-43e3-98a4-2c0cd69b6b5f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.149198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1127/2269] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:07.154987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218ad08-4cdf-4f78-bbb7-10d6053a28a5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.155274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218ad08-4cdf-4f78-bbb7-10d6053a28a5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.155546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218ad08-4cdf-4f78-bbb7-10d6053a28a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.180214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c218ad08-4cdf-4f78-bbb7-10d6053a28a5 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:07.181624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c218ad08-4cdf-4f78-bbb7-10d6053a28a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:07.191974 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1143/2270] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:07:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:07:07.204645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205f3b9-610f-43e3-98a4-2c0cd69b6b5f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:07.207820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d205f3b9-610f-43e3-98a4-2c0cd69b6b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:07.222333 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205f3b9-610f-43e3-98a4-2c0cd69b6b5f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:07.225979 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4a073-aec1-4fbe-a586-57bcdd14e1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:07.226676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4a073-aec1-4fbe-a586-57bcdd14e1da service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.226964 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4a073-aec1-4fbe-a586-57bcdd14e1da service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.227242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4a073-aec1-4fbe-a586-57bcdd14e1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.239799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:07.242206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4a073-aec1-4fbe-a586-57bcdd14e1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:07.243771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4a073-aec1-4fbe-a586-57bcdd14e1da service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:07.259298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1128/2271] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:07:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:07:13.540988 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41abd6b-b87f-4421-9174-17c4df7c31fd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:13.541505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41abd6b-b87f-4421-9174-17c4df7c31fd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:13.541600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41abd6b-b87f-4421-9174-17c4df7c31fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:13.558553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e41abd6b-b87f-4421-9174-17c4df7c31fd None None] MFA Rules not processed for user `4e6db41ec3b94c27bb4af92a35ee0e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:13.559738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e41abd6b-b87f-4421-9174-17c4df7c31fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:13.569122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:13.618090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1144/2272] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:07:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:14.628564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e12a44-898b-4632-bae3-88a4e7d998f1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:14.629148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e12a44-898b-4632-bae3-88a4e7d998f1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:14.629148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e12a44-898b-4632-bae3-88a4e7d998f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:14.678386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-78e12a44-898b-4632-bae3-88a4e7d998f1 None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:14.679615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78e12a44-898b-4632-bae3-88a4e7d998f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:14.751152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1129/2273] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:14.764621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41abd6b-b87f-4421-9174-17c4df7c31fd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:14.766074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e41abd6b-b87f-4421-9174-17c4df7c31fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:14.861516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41abd6b-b87f-4421-9174-17c4df7c31fd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:14.866735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90bad9-3f76-427d-98d2-70eb6b4a3265 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:14.867390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90bad9-3f76-427d-98d2-70eb6b4a3265 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:14.867802 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90bad9-3f76-427d-98d2-70eb6b4a3265 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:14.868300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90bad9-3f76-427d-98d2-70eb6b4a3265 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:14.878454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90bad9-3f76-427d-98d2-70eb6b4a3265 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:14.880066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90bad9-3f76-427d-98d2-70eb6b4a3265 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:14.888475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1145/2274] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:07:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:07:24.747811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=42 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:24.885004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=59 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:29.522169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e12a44-898b-4632-bae3-88a4e7d998f1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.583255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e12a44-898b-4632-bae3-88a4e7d998f1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:29.587624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:29.588479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.588760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.589035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] PATH_INFO: `/v3/users/ce5b08d2b78b482380db0502873c1e23` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.590211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5b08d2b78b482380db0502873c1e23)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:29.592170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:29.637180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5b08d2b78b482380db0502873c1e23'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:07:29.637818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5b08d2b78b482380db0502873c1e23'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:07:29.644676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5b08d2b78b482380db0502873c1e23'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:07:29.645283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5b08d2b78b482380db0502873c1e23'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:07:29.657952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5b08d2b78b482380db0502873c1e23'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:07:29.670307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1130/2275] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:07:29 2022] DELETE /identity/v3/users/ce5b08d2b78b482380db0502873c1e23 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:29.674765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5003c8e-1e33-49ab-a2fb-1ac6be25300d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.675258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5003c8e-1e33-49ab-a2fb-1ac6be25300d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.675367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5003c8e-1e33-49ab-a2fb-1ac6be25300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.693710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f5003c8e-1e33-49ab-a2fb-1ac6be25300d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:29.694929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f5003c8e-1e33-49ab-a2fb-1ac6be25300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:29.751838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1146/2276] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:29.761411 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.762867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:29.823607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953aa53c-ada9-475b-ab98-81002b32193d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:29.827272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07be7d2-4434-49bf-a176-2d538162b6f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:29.827941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07be7d2-4434-49bf-a176-2d538162b6f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.828221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07be7d2-4434-49bf-a176-2d538162b6f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.828496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07be7d2-4434-49bf-a176-2d538162b6f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.837423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07be7d2-4434-49bf-a176-2d538162b6f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:29.838901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07be7d2-4434-49bf-a176-2d538162b6f7 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:29.849182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1131/2277] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:07:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:07:30.309331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5003c8e-1e33-49ab-a2fb-1ac6be25300d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:30.310779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f5003c8e-1e33-49ab-a2fb-1ac6be25300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:30.322648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5003c8e-1e33-49ab-a2fb-1ac6be25300d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:30.327378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:30.328237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.328519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.328796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] PATH_INFO: `/v3/projects/35fbec6984b044b889e81fbba81ce8d6` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.330029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35fbec6984b044b889e81fbba81ce8d6)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:30.332645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:30.378983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35fbec6984b044b889e81fbba81ce8d6', 'request_id': 'req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:07:30.384982 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35fbec6984b044b889e81fbba81ce8d6', 'request_id': 'req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:07:30.390501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35fbec6984b044b889e81fbba81ce8d6', 'request_id': 'req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:07:30.402996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] The token cache is being invalidate because project 35fbec6984b044b889e81fbba81ce8d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:07:30.403282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:07:30.403885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-5d8ded17-1976-45d2-b2fb-4cab9c4eec03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:07:30.405414 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1147/2278] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:07:30 2022] DELETE /identity/v3/projects/35fbec6984b044b889e81fbba81ce8d6 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:30.411155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985be187-56b3-43aa-921e-84a3a07967b7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.411438 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985be187-56b3-43aa-921e-84a3a07967b7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.411712 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985be187-56b3-43aa-921e-84a3a07967b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.428959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-985be187-56b3-43aa-921e-84a3a07967b7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:30.430148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-985be187-56b3-43aa-921e-84a3a07967b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:30.446892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:30.500036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1132/2279] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:30.508426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f175b-6e4d-4637-a853-5c6be57a19f4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.508716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f175b-6e4d-4637-a853-5c6be57a19f4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.508995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f175b-6e4d-4637-a853-5c6be57a19f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.528000 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-792f175b-6e4d-4637-a853-5c6be57a19f4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:30.529199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-792f175b-6e4d-4637-a853-5c6be57a19f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:30.537152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1148/2280] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:30.541277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985be187-56b3-43aa-921e-84a3a07967b7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:30.551440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985be187-56b3-43aa-921e-84a3a07967b7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:30.559961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caecbcb-ac9e-4416-954f-5c56a1528300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:30.560721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caecbcb-ac9e-4416-954f-5c56a1528300 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.561006 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caecbcb-ac9e-4416-954f-5c56a1528300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.561284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caecbcb-ac9e-4416-954f-5c56a1528300 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.562022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caecbcb-ac9e-4416-954f-5c56a1528300 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:30.562341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caecbcb-ac9e-4416-954f-5c56a1528300 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:30.564160 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caecbcb-ac9e-4416-954f-5c56a1528300 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:30.576511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1133/2281] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:30.580816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f175b-6e4d-4637-a853-5c6be57a19f4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:30.590946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f175b-6e4d-4637-a853-5c6be57a19f4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:30.595682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f17c19-c1f4-4fed-ad2a-a8187ac7ef3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:30.596347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f17c19-c1f4-4fed-ad2a-a8187ac7ef3b admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.596632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f17c19-c1f4-4fed-ad2a-a8187ac7ef3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.596910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f17c19-c1f4-4fed-ad2a-a8187ac7ef3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.597658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f17c19-c1f4-4fed-ad2a-a8187ac7ef3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:30.599658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f17c19-c1f4-4fed-ad2a-a8187ac7ef3b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:30.624844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1149/2282] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:07:30 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:07:30.629394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caecbcb-ac9e-4416-954f-5c56a1528300 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:30.639769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caecbcb-ac9e-4416-954f-5c56a1528300 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:30.696468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489551-2351-40f3-bec2-4503d0f9fa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:30.697136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489551-2351-40f3-bec2-4503d0f9fa59 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.697418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489551-2351-40f3-bec2-4503d0f9fa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.697692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5489551-2351-40f3-bec2-4503d0f9fa59 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.698378 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5489551-2351-40f3-bec2-4503d0f9fa59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:30.700371 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5489551-2351-40f3-bec2-4503d0f9fa59 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:30.719622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1134/2283] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:07:30 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:30.723894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f17c19-c1f4-4fed-ad2a-a8187ac7ef3b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:30.734117 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f17c19-c1f4-4fed-ad2a-a8187ac7ef3b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:30.738747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07f143-e77f-481f-84e6-46cf3562146b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:30.739397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07f143-e77f-481f-84e6-46cf3562146b admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.739679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07f143-e77f-481f-84e6-46cf3562146b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.739953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07f143-e77f-481f-84e6-46cf3562146b admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.740665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07f143-e77f-481f-84e6-46cf3562146b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:30.742250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07f143-e77f-481f-84e6-46cf3562146b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:30.761501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1150/2284] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:07:30.765560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489551-2351-40f3-bec2-4503d0f9fa59 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:30.776342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5489551-2351-40f3-bec2-4503d0f9fa59 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:30.780961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83714513-9e11-4176-ae97-8bb1eb5ce0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:30.781643 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83714513-9e11-4176-ae97-8bb1eb5ce0da admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.781927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83714513-9e11-4176-ae97-8bb1eb5ce0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.782206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83714513-9e11-4176-ae97-8bb1eb5ce0da admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.782994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83714513-9e11-4176-ae97-8bb1eb5ce0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:30.784602 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83714513-9e11-4176-ae97-8bb1eb5ce0da admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:30.794374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1135/2285] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:07:30.798422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07f143-e77f-481f-84e6-46cf3562146b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:30.808569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07f143-e77f-481f-84e6-46cf3562146b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:30.813329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cc810-95d5-417c-8bf7-63ef1c7ddd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:30.814153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cc810-95d5-417c-8bf7-63ef1c7ddd17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.814433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cc810-95d5-417c-8bf7-63ef1c7ddd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.814744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cc810-95d5-417c-8bf7-63ef1c7ddd17 admin admin] PATH_INFO: `/v3/projects/cdb0cbaf75ee402091c3eec9dd512a2d/users/6917bcbeaa204b159efc6cf456465c2a/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.829981 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cc810-95d5-417c-8bf7-63ef1c7ddd17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdb0cbaf75ee402091c3eec9dd512a2d, user_id=6917bcbeaa204b159efc6cf456465c2a, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:30.833352 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cc810-95d5-417c-8bf7-63ef1c7ddd17 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:30.846420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1151/2286] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:07:30 2022] PUT /identity/v3/projects/cdb0cbaf75ee402091c3eec9dd512a2d/users/6917bcbeaa204b159efc6cf456465c2a/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:30.851501 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78c7ba-63aa-47dc-b7e0-906a6dd53c79 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.851791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78c7ba-63aa-47dc-b7e0-906a6dd53c79 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.852071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78c7ba-63aa-47dc-b7e0-906a6dd53c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.870581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9c78c7ba-63aa-47dc-b7e0-906a6dd53c79 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:30.871774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9c78c7ba-63aa-47dc-b7e0-906a6dd53c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:30.929742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1136/2287] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:07:30.939263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cc810-95d5-417c-8bf7-63ef1c7ddd17 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:30.940904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9c1cc810-95d5-417c-8bf7-63ef1c7ddd17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:30.963261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:31.002116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cc810-95d5-417c-8bf7-63ef1c7ddd17 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:31.006391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014360a9-aacc-4290-a623-b0b2a793c455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:31.007064 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014360a9-aacc-4290-a623-b0b2a793c455 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:31.007354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014360a9-aacc-4290-a623-b0b2a793c455 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:31.007635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014360a9-aacc-4290-a623-b0b2a793c455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:31.020657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014360a9-aacc-4290-a623-b0b2a793c455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:31.022103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014360a9-aacc-4290-a623-b0b2a793c455 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:31.032056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1152/2288] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:07:35.417348 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972347cc-1ae4-4085-ada5-a1c10ba9d8e3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.417814 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972347cc-1ae4-4085-ada5-a1c10ba9d8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.417930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972347cc-1ae4-4085-ada5-a1c10ba9d8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.449129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-972347cc-1ae4-4085-ada5-a1c10ba9d8e3 None None] MFA Rules not processed for user `6917bcbeaa204b159efc6cf456465c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:35.450469 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-972347cc-1ae4-4085-ada5-a1c10ba9d8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:35.470848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:35.523936 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1137/2289] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:07:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:35.528342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014360a9-aacc-4290-a623-b0b2a793c455 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:35.538305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014360a9-aacc-4290-a623-b0b2a793c455 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:35.542903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7c708-a855-47a5-a6fc-8444af5dd423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:35.543553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7c708-a855-47a5-a6fc-8444af5dd423 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.543840 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7c708-a855-47a5-a6fc-8444af5dd423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.544121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7c708-a855-47a5-a6fc-8444af5dd423 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.544811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7c708-a855-47a5-a6fc-8444af5dd423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:35.546737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7c708-a855-47a5-a6fc-8444af5dd423 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:35.572614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1153/2290] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:07:35 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:35.576621 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972347cc-1ae4-4085-ada5-a1c10ba9d8e3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:35.587159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972347cc-1ae4-4085-ada5-a1c10ba9d8e3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:35.642605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f7834-87b9-48c6-a815-18ab02779f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:35.643268 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7834-87b9-48c6-a815-18ab02779f1d admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.643552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7834-87b9-48c6-a815-18ab02779f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.643830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7834-87b9-48c6-a815-18ab02779f1d admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.644550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f7834-87b9-48c6-a815-18ab02779f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:35.646564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f7834-87b9-48c6-a815-18ab02779f1d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:35.666554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1138/2291] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:07:35 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:35.670515 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7c708-a855-47a5-a6fc-8444af5dd423 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:35.683235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7c708-a855-47a5-a6fc-8444af5dd423 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:35.688817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be7c24-d5e1-445f-913f-cd82c723650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:35.689494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be7c24-d5e1-445f-913f-cd82c723650c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.689782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be7c24-d5e1-445f-913f-cd82c723650c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.690171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be7c24-d5e1-445f-913f-cd82c723650c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.690910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be7c24-d5e1-445f-913f-cd82c723650c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:35.693115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16be7c24-d5e1-445f-913f-cd82c723650c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:35.704516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1154/2292] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:07:35.708684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f7834-87b9-48c6-a815-18ab02779f1d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:35.718717 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f7834-87b9-48c6-a815-18ab02779f1d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:35.723740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1200bd-d1cf-4f94-8f0b-d455c5e4a05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:35.724596 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1200bd-d1cf-4f94-8f0b-d455c5e4a05d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.724875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1200bd-d1cf-4f94-8f0b-d455c5e4a05d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.725152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1200bd-d1cf-4f94-8f0b-d455c5e4a05d admin admin] PATH_INFO: `/v3/projects/063d0056b740469fb8152f77df552a6a/users/43ed9f89c496438994c24baaf990062c/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.740229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1200bd-d1cf-4f94-8f0b-d455c5e4a05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=063d0056b740469fb8152f77df552a6a, user_id=43ed9f89c496438994c24baaf990062c, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:35.742977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1200bd-d1cf-4f94-8f0b-d455c5e4a05d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:35.756103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1139/2293] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:07:35 2022] PUT /identity/v3/projects/063d0056b740469fb8152f77df552a6a/users/43ed9f89c496438994c24baaf990062c/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:35.760196 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be7c24-d5e1-445f-913f-cd82c723650c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:35.770389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be7c24-d5e1-445f-913f-cd82c723650c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:35.827141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfbd9c-7847-454b-9962-4355499ae0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:35.827805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfbd9c-7847-454b-9962-4355499ae0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.828085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfbd9c-7847-454b-9962-4355499ae0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.828358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfbd9c-7847-454b-9962-4355499ae0f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.829068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfbd9c-7847-454b-9962-4355499ae0f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:35.830688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dfbd9c-7847-454b-9962-4355499ae0f7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:35.840221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1155/2294] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:07:35.844272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1200bd-d1cf-4f94-8f0b-d455c5e4a05d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:35.854944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1200bd-d1cf-4f94-8f0b-d455c5e4a05d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:35.859537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7185b8-1de3-4e7b-a8cf-651bb10b5949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:35.860403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7185b8-1de3-4e7b-a8cf-651bb10b5949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.860684 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7185b8-1de3-4e7b-a8cf-651bb10b5949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.860961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7185b8-1de3-4e7b-a8cf-651bb10b5949 admin admin] PATH_INFO: `/v3/projects/063d0056b740469fb8152f77df552a6a/users/43ed9f89c496438994c24baaf990062c/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.862994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7185b8-1de3-4e7b-a8cf-651bb10b5949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063d0056b740469fb8152f77df552a6a, user_id=43ed9f89c496438994c24baaf990062c, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:35.866167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7185b8-1de3-4e7b-a8cf-651bb10b5949 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:35.877185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1140/2295] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:07:35 2022] PUT /identity/v3/projects/063d0056b740469fb8152f77df552a6a/users/43ed9f89c496438994c24baaf990062c/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:40.941590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28155a4-92b4-4005-ab51-8dd3e3863dfd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.942136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28155a4-92b4-4005-ab51-8dd3e3863dfd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.942136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28155a4-92b4-4005-ab51-8dd3e3863dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.973469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c28155a4-92b4-4005-ab51-8dd3e3863dfd None None] MFA Rules not processed for user `43ed9f89c496438994c24baaf990062c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:40.974674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c28155a4-92b4-4005-ab51-8dd3e3863dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:41.045730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1156/2296] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:07:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:41.058477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7185b8-1de3-4e7b-a8cf-651bb10b5949 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.060211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4e7185b8-1de3-4e7b-a8cf-651bb10b5949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:41.134654 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7185b8-1de3-4e7b-a8cf-651bb10b5949 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.138263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b285a3-9fce-492a-a13f-a8fc8b5a2191 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3bd8dd462846bf9423f8528aa5c489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.138935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b285a3-9fce-492a-a13f-a8fc8b5a2191 service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.139222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b285a3-9fce-492a-a13f-a8fc8b5a2191 service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.139506 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b285a3-9fce-492a-a13f-a8fc8b5a2191 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.152114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:41.199918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b285a3-9fce-492a-a13f-a8fc8b5a2191 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.201374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b285a3-9fce-492a-a13f-a8fc8b5a2191 service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.210293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1141/2297] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:07:41.306674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77951d2d-a402-401b-8c6d-a88c873a15db None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.306977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77951d2d-a402-401b-8c6d-a88c873a15db None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.307261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77951d2d-a402-401b-8c6d-a88c873a15db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.325406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-77951d2d-a402-401b-8c6d-a88c873a15db None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:41.326670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-77951d2d-a402-401b-8c6d-a88c873a15db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:41.331985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:41.383091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1157/2298] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:41.404496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b285a3-9fce-492a-a13f-a8fc8b5a2191 service glance] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.405847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c1b285a3-9fce-492a-a13f-a8fc8b5a2191 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:41.481442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b285a3-9fce-492a-a13f-a8fc8b5a2191 service glance] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.487625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89312d-613d-4e2d-906b-b9a0bf24694b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.488280 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89312d-613d-4e2d-906b-b9a0bf24694b service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.488561 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89312d-613d-4e2d-906b-b9a0bf24694b service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.488837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89312d-613d-4e2d-906b-b9a0bf24694b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.497512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89312d-613d-4e2d-906b-b9a0bf24694b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.499151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89312d-613d-4e2d-906b-b9a0bf24694b service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.508026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1142/2299] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:07:48.443171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3691d-1a44-4cf9-a5cb-b0ea5b90c722 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.443772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3691d-1a44-4cf9-a5cb-b0ea5b90c722 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.443772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3691d-1a44-4cf9-a5cb-b0ea5b90c722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.463316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-65b3691d-1a44-4cf9-a5cb-b0ea5b90c722 None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:48.465124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-65b3691d-1a44-4cf9-a5cb-b0ea5b90c722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:48.476629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1158/2300] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 26 21:07:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:48.495361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89312d-613d-4e2d-906b-b9a0bf24694b service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.508380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89312d-613d-4e2d-906b-b9a0bf24694b service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.515722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97866c9-bcc9-44eb-aed1-891f32ee69a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.516804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97866c9-bcc9-44eb-aed1-891f32ee69a2 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.516804 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97866c9-bcc9-44eb-aed1-891f32ee69a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.516999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97866c9-bcc9-44eb-aed1-891f32ee69a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.527749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97866c9-bcc9-44eb-aed1-891f32ee69a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.529421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97866c9-bcc9-44eb-aed1-891f32ee69a2 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.539435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1143/2301] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:07:52.030284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c0d0e-f242-4cf8-8e92-5220627e2607 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.030841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c0d0e-f242-4cf8-8e92-5220627e2607 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.030936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c0d0e-f242-4cf8-8e92-5220627e2607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.048458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-cc4c0d0e-f242-4cf8-8e92-5220627e2607 None None] MFA Rules not processed for user `6917bcbeaa204b159efc6cf456465c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:52.049674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cc4c0d0e-f242-4cf8-8e92-5220627e2607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:52.058032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1159/2302] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:07:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:53.044893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97866c9-bcc9-44eb-aed1-891f32ee69a2 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:53.046388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b97866c9-bcc9-44eb-aed1-891f32ee69a2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:53.118905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97866c9-bcc9-44eb-aed1-891f32ee69a2 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:53.124379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b9f55-3f32-4de4-8d43-f14320568863 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:53.125026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9f55-3f32-4de4-8d43-f14320568863 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.125308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9f55-3f32-4de4-8d43-f14320568863 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.125587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b9f55-3f32-4de4-8d43-f14320568863 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.127627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6c0b9f55-3f32-4de4-8d43-f14320568863 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:53.140829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b9f55-3f32-4de4-8d43-f14320568863 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:53.142440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b9f55-3f32-4de4-8d43-f14320568863 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:53.153077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1144/2303] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:07:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:07:54.603296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea58a6-c887-4a5f-bb4f-3b134bec92db None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.604050 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea58a6-c887-4a5f-bb4f-3b134bec92db None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.604341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea58a6-c887-4a5f-bb4f-3b134bec92db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.623853 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-96ea58a6-c887-4a5f-bb4f-3b134bec92db None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:54.625160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-96ea58a6-c887-4a5f-bb4f-3b134bec92db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:54.633688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1160/2304] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:07:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:07:54.650363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b9f55-3f32-4de4-8d43-f14320568863 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.663841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b9f55-3f32-4de4-8d43-f14320568863 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.669484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508c16f-0e33-46fe-a710-34b446fc7f89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.670248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508c16f-0e33-46fe-a710-34b446fc7f89 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.670578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508c16f-0e33-46fe-a710-34b446fc7f89 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.670861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508c16f-0e33-46fe-a710-34b446fc7f89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.680770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508c16f-0e33-46fe-a710-34b446fc7f89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:54.682661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508c16f-0e33-46fe-a710-34b446fc7f89 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:54.702624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1145/2305] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:08:04.620466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=29 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:04.691808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:27.808993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea58a6-c887-4a5f-bb4f-3b134bec92db None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:27.810417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-96ea58a6-c887-4a5f-bb4f-3b134bec92db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:27.865126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea58a6-c887-4a5f-bb4f-3b134bec92db None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.869498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2c621-5ac0-4027-bfa9-289dbefffced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:27.870151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2c621-5ac0-4027-bfa9-289dbefffced service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.870480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2c621-5ac0-4027-bfa9-289dbefffced service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.870750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2c621-5ac0-4027-bfa9-289dbefffced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.880004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2c621-5ac0-4027-bfa9-289dbefffced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:27.882087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2c621-5ac0-4027-bfa9-289dbefffced service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:27.891560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1161/2306] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:08:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:08:29.749687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00615-cfb8-4f00-8fb0-b1cd6becb358 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:29.751327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00615-cfb8-4f00-8fb0-b1cd6becb358 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:29.751704 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00615-cfb8-4f00-8fb0-b1cd6becb358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:29.775663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8fa00615-cfb8-4f00-8fb0-b1cd6becb358 None None] MFA Rules not processed for user `443be2bc27514432a34d2a4c57108b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:29.776946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8fa00615-cfb8-4f00-8fb0-b1cd6becb358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:29.909276 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1146/2307] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:08:37.887301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=26 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:39.893140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=25 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:43.669392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dab26-bea3-49ea-b9f6-3676cd87d42d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:43.670971 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dab26-bea3-49ea-b9f6-3676cd87d42d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:43.671263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dab26-bea3-49ea-b9f6-3676cd87d42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:43.689535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-240dab26-bea3-49ea-b9f6-3676cd87d42d None None] MFA Rules not processed for user `8145168b1d02438886cd0865844f7480`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:43.690767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-240dab26-bea3-49ea-b9f6-3676cd87d42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:43.746196 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1162/2308] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:08:50.696765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c282021-9112-4ea6-a945-825af606145c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:50.697301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c282021-9112-4ea6-a945-825af606145c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:50.697396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c282021-9112-4ea6-a945-825af606145c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:50.715230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4c282021-9112-4ea6-a945-825af606145c None None] MFA Rules not processed for user `4e6db41ec3b94c27bb4af92a35ee0e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:50.716439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4c282021-9112-4ea6-a945-825af606145c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:50.775930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1147/2309] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:08:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:08:53.742815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=25 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:00.768714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=25 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:01.134953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c40091-f3b6-4f28-877b-72c1882c11a7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:01.135462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c40091-f3b6-4f28-877b-72c1882c11a7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:01.135557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c40091-f3b6-4f28-877b-72c1882c11a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:01.153172 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-75c40091-f3b6-4f28-877b-72c1882c11a7 None None] MFA Rules not processed for user `83f3cfb67d3147d7b856158dcf32ff03`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:01.154396 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-75c40091-f3b6-4f28-877b-72c1882c11a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:01.210672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1163/2310] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:11.203969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=25 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:21.840325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c282021-9112-4ea6-a945-825af606145c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:21.850410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c282021-9112-4ea6-a945-825af606145c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:21.854062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10094b-453d-4aa5-a99e-e880b711aae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:21.854755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10094b-453d-4aa5-a99e-e880b711aae8 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:21.855039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10094b-453d-4aa5-a99e-e880b711aae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:21.855316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10094b-453d-4aa5-a99e-e880b711aae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:21.857094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dd10094b-453d-4aa5-a99e-e880b711aae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:21.926573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10094b-453d-4aa5-a99e-e880b711aae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:21.928572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd10094b-453d-4aa5-a99e-e880b711aae8 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:21.939362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1148/2311] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:23.105863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c40091-f3b6-4f28-877b-72c1882c11a7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:23.115769 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c40091-f3b6-4f28-877b-72c1882c11a7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:23.119578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b04ec7-729a-4879-9092-67b7a8e7df3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:23.120207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b04ec7-729a-4879-9092-67b7a8e7df3f service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:23.120493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b04ec7-729a-4879-9092-67b7a8e7df3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:23.120770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b04ec7-729a-4879-9092-67b7a8e7df3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:23.122536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c4b04ec7-729a-4879-9092-67b7a8e7df3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:23.136076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b04ec7-729a-4879-9092-67b7a8e7df3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:23.137496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b04ec7-729a-4879-9092-67b7a8e7df3f service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:23.148358 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1164/2312] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:31.934596 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=34 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:33.143603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:41.360465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10094b-453d-4aa5-a99e-e880b711aae8 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:41.361907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dd10094b-453d-4aa5-a99e-e880b711aae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:41.376671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd10094b-453d-4aa5-a99e-e880b711aae8 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:41.382193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cd9ef-cfc7-4483-858e-31107739bda7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:41.382867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd9ef-cfc7-4483-858e-31107739bda7 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:41.383148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd9ef-cfc7-4483-858e-31107739bda7 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:41.383419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cd9ef-cfc7-4483-858e-31107739bda7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:41.385149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-356cd9ef-cfc7-4483-858e-31107739bda7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:41.395234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cd9ef-cfc7-4483-858e-31107739bda7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:41.396812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cd9ef-cfc7-4483-858e-31107739bda7 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:41.405590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1149/2313] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:41.448492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b04ec7-729a-4879-9092-67b7a8e7df3f service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:41.457019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b04ec7-729a-4879-9092-67b7a8e7df3f service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:41.460614 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc99eef-1f25-4409-ae5d-a56687ca0f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:41.461242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc99eef-1f25-4409-ae5d-a56687ca0f84 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:41.461545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc99eef-1f25-4409-ae5d-a56687ca0f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:41.461799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc99eef-1f25-4409-ae5d-a56687ca0f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:41.463582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2bc99eef-1f25-4409-ae5d-a56687ca0f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:41.525318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc99eef-1f25-4409-ae5d-a56687ca0f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:41.526717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc99eef-1f25-4409-ae5d-a56687ca0f84 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:41.536857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1165/2314] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:43.006852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cd9ef-cfc7-4483-858e-31107739bda7 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:43.008394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-356cd9ef-cfc7-4483-858e-31107739bda7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:43.024545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cd9ef-cfc7-4483-858e-31107739bda7 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:43.030849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb411e25-57b6-4f94-91b4-758fd5daad2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:43.031529 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb411e25-57b6-4f94-91b4-758fd5daad2b service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:43.031801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb411e25-57b6-4f94-91b4-758fd5daad2b service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:43.032077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb411e25-57b6-4f94-91b4-758fd5daad2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:43.033837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-eb411e25-57b6-4f94-91b4-758fd5daad2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:43.045090 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb411e25-57b6-4f94-91b4-758fd5daad2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:43.046709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb411e25-57b6-4f94-91b4-758fd5daad2b service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:43.055913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1150/2315] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:45.136190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc99eef-1f25-4409-ae5d-a56687ca0f84 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:45.137675 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2bc99eef-1f25-4409-ae5d-a56687ca0f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:45.153164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc99eef-1f25-4409-ae5d-a56687ca0f84 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:45.157832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:45.158730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:45.159014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:45.159292 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] PATH_INFO: `/v3/users/443be2bc27514432a34d2a4c57108b7a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:45.160461 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] RBAC: Authorizing `identity:delete_user(user_id=443be2bc27514432a34d2a4c57108b7a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:45.162403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:45.211009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443be2bc27514432a34d2a4c57108b7a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:45.223792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443be2bc27514432a34d2a4c57108b7a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:45.231852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443be2bc27514432a34d2a4c57108b7a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:45.232666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443be2bc27514432a34d2a4c57108b7a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:45.239975 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a477f5e4-550d-4a51-8ab6-026faa06876d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443be2bc27514432a34d2a4c57108b7a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:45.246554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1166/2316] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:09:45 2022] DELETE /identity/v3/users/443be2bc27514432a34d2a4c57108b7a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:45.418919 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb411e25-57b6-4f94-91b4-758fd5daad2b service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:45.430714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb411e25-57b6-4f94-91b4-758fd5daad2b service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:45.491201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:45.492061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:45.492607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:45.492882 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] PATH_INFO: `/v3/projects/9f8ac962c5d0402eb17b01aa02632897` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:45.494176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f8ac962c5d0402eb17b01aa02632897)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:45.496141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:45.547766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f8ac962c5d0402eb17b01aa02632897', 'request_id': 'req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:45.554209 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f8ac962c5d0402eb17b01aa02632897', 'request_id': 'req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:45.562828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f8ac962c5d0402eb17b01aa02632897', 'request_id': 'req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:45.576594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] The token cache is being invalidate because project 9f8ac962c5d0402eb17b01aa02632897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:09:45.576887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:45.577543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7bf06b-1c0f-41f6-a58b-2a058f08178c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:45.579172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1151/2317] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:09:45 2022] DELETE /identity/v3/projects/9f8ac962c5d0402eb17b01aa02632897 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:45.584020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07068944-9fc0-4c25-bfaf-a3402102e9fd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:45.584316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07068944-9fc0-4c25-bfaf-a3402102e9fd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:45.584617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07068944-9fc0-4c25-bfaf-a3402102e9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:45.602774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-07068944-9fc0-4c25-bfaf-a3402102e9fd None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:45.603982 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-07068944-9fc0-4c25-bfaf-a3402102e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:45.666588 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1167/2318] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:45.674498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67f1d4-5e5d-4640-a778-9373e5916de5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:45.674817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67f1d4-5e5d-4640-a778-9373e5916de5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:45.675077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67f1d4-5e5d-4640-a778-9373e5916de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:45.694394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ee67f1d4-5e5d-4640-a778-9373e5916de5 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:45.695636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ee67f1d4-5e5d-4640-a778-9373e5916de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:45.703701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1152/2319] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:45.707893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07068944-9fc0-4c25-bfaf-a3402102e9fd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:45.717596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07068944-9fc0-4c25-bfaf-a3402102e9fd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:45.722150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a965b16-1eb3-4bc7-b00a-788ae21e9c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:45.722835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a965b16-1eb3-4bc7-b00a-788ae21e9c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:45.723122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a965b16-1eb3-4bc7-b00a-788ae21e9c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:45.723920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a965b16-1eb3-4bc7-b00a-788ae21e9c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:45.724639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a965b16-1eb3-4bc7-b00a-788ae21e9c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:45.724948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a965b16-1eb3-4bc7-b00a-788ae21e9c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:45.726570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a965b16-1eb3-4bc7-b00a-788ae21e9c69 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:45.728028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:45.740498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1168/2320] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:45.744839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67f1d4-5e5d-4640-a778-9373e5916de5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:45.754004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67f1d4-5e5d-4640-a778-9373e5916de5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:45.758828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fdbd0-0553-4a54-9f53-aedb72ae163b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:45.759471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fdbd0-0553-4a54-9f53-aedb72ae163b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:45.759757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fdbd0-0553-4a54-9f53-aedb72ae163b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:45.760035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fdbd0-0553-4a54-9f53-aedb72ae163b admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:45.760715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fdbd0-0553-4a54-9f53-aedb72ae163b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:45.762659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fdbd0-0553-4a54-9f53-aedb72ae163b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:45.787705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1153/2321] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:09:45 2022] POST /identity/v3/projects => generated 363 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:45.791682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a965b16-1eb3-4bc7-b00a-788ae21e9c69 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:45.802352 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a965b16-1eb3-4bc7-b00a-788ae21e9c69 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:45.857553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b585e1-5e1c-457f-951d-4ebe3f21540a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:45.858254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b585e1-5e1c-457f-951d-4ebe3f21540a admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:45.858568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b585e1-5e1c-457f-951d-4ebe3f21540a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:45.858850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b585e1-5e1c-457f-951d-4ebe3f21540a admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:45.859536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b585e1-5e1c-457f-951d-4ebe3f21540a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:45.861447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b585e1-5e1c-457f-951d-4ebe3f21540a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:45.881195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1169/2322] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:09:45 2022] POST /identity/v3/users => generated 282 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:09:45.885407 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fdbd0-0553-4a54-9f53-aedb72ae163b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:45.896039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fdbd0-0553-4a54-9f53-aedb72ae163b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:45.900745 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b56d1-7515-4ec4-a3c5-4ffe17b0cc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:45.901394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b56d1-7515-4ec4-a3c5-4ffe17b0cc1a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:45.901680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b56d1-7515-4ec4-a3c5-4ffe17b0cc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:45.901962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b56d1-7515-4ec4-a3c5-4ffe17b0cc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:45.902723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b56d1-7515-4ec4-a3c5-4ffe17b0cc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:45.904398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b56d1-7515-4ec4-a3c5-4ffe17b0cc1a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:45.915114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1154/2323] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:45.918882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b585e1-5e1c-457f-951d-4ebe3f21540a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:45.929401 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b585e1-5e1c-457f-951d-4ebe3f21540a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:45.934185 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a806bbd-2c01-46b2-bbca-8a2fec8398f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:45.934886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a806bbd-2c01-46b2-bbca-8a2fec8398f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:45.935174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a806bbd-2c01-46b2-bbca-8a2fec8398f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:45.935456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a806bbd-2c01-46b2-bbca-8a2fec8398f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:45.936179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a806bbd-2c01-46b2-bbca-8a2fec8398f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:45.937774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a806bbd-2c01-46b2-bbca-8a2fec8398f7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:45.948342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1170/2324] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:45.954219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b56d1-7515-4ec4-a3c5-4ffe17b0cc1a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:45.965172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b56d1-7515-4ec4-a3c5-4ffe17b0cc1a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:45.969924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30cdce-9ca2-4bf6-bc93-43b01b6875bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:45.970909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30cdce-9ca2-4bf6-bc93-43b01b6875bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:45.971464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30cdce-9ca2-4bf6-bc93-43b01b6875bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:45.971820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30cdce-9ca2-4bf6-bc93-43b01b6875bb admin admin] PATH_INFO: `/v3/projects/1f272deab70b40d49f1d6ca517458be6/users/2f8aec4480264b8da80ae7c9a49e0a9f/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:45.991565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30cdce-9ca2-4bf6-bc93-43b01b6875bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f272deab70b40d49f1d6ca517458be6, user_id=2f8aec4480264b8da80ae7c9a49e0a9f, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:45.995005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30cdce-9ca2-4bf6-bc93-43b01b6875bb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:46.006544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:46.012646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1155/2325] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:09:45 2022] PUT /identity/v3/projects/1f272deab70b40d49f1d6ca517458be6/users/2f8aec4480264b8da80ae7c9a49e0a9f/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:46.023207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca6a2c-cc18-4cf3-b3e9-90be788c9618 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:46.023573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca6a2c-cc18-4cf3-b3e9-90be788c9618 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:46.023927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca6a2c-cc18-4cf3-b3e9-90be788c9618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:46.053942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-96ca6a2c-cc18-4cf3-b3e9-90be788c9618 None None] MFA Rules not processed for user `2f8aec4480264b8da80ae7c9a49e0a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:46.055131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-96ca6a2c-cc18-4cf3-b3e9-90be788c9618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:46.122318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1171/2326] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:46 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:46.126766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30cdce-9ca2-4bf6-bc93-43b01b6875bb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:46.192825 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30cdce-9ca2-4bf6-bc93-43b01b6875bb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:46.197569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83cd44-bb17-4a32-bf58-7f5e5349b522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:46.198226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83cd44-bb17-4a32-bf58-7f5e5349b522 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:46.198553 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83cd44-bb17-4a32-bf58-7f5e5349b522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:46.198838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83cd44-bb17-4a32-bf58-7f5e5349b522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:46.199578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83cd44-bb17-4a32-bf58-7f5e5349b522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:46.201513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83cd44-bb17-4a32-bf58-7f5e5349b522 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:46.230228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1156/2327] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:09:46 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:46.231416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca6a2c-cc18-4cf3-b3e9-90be788c9618 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:46.242048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca6a2c-cc18-4cf3-b3e9-90be788c9618 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:46.297380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af571e6-63aa-4580-9570-d9ce398700ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:46.298035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af571e6-63aa-4580-9570-d9ce398700ac admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:46.298318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af571e6-63aa-4580-9570-d9ce398700ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:46.298763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af571e6-63aa-4580-9570-d9ce398700ac admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:46.299466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af571e6-63aa-4580-9570-d9ce398700ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:46.301354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af571e6-63aa-4580-9570-d9ce398700ac admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:46.308879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:46.321031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1172/2328] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:09:46 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:09:46.325123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83cd44-bb17-4a32-bf58-7f5e5349b522 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:46.335928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83cd44-bb17-4a32-bf58-7f5e5349b522 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:46.340543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d751f-72ed-4a4b-974b-c40588f83028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:46.341200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d751f-72ed-4a4b-974b-c40588f83028 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:46.341489 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d751f-72ed-4a4b-974b-c40588f83028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:46.341768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d751f-72ed-4a4b-974b-c40588f83028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:46.342525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d751f-72ed-4a4b-974b-c40588f83028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:46.344144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d751f-72ed-4a4b-974b-c40588f83028 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:46.355226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1157/2329] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:46.359218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af571e6-63aa-4580-9570-d9ce398700ac admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:46.369411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af571e6-63aa-4580-9570-d9ce398700ac admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:46.374057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8624634-7fe8-4e6e-8d79-db34fe9ea437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:46.374975 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8624634-7fe8-4e6e-8d79-db34fe9ea437 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:46.375260 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8624634-7fe8-4e6e-8d79-db34fe9ea437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:46.375538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8624634-7fe8-4e6e-8d79-db34fe9ea437 admin admin] PATH_INFO: `/v3/projects/ed404e2262d44a3294d19ca3dd263134/users/c13aa0bc422d4e8081bc704b437f5f52/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:46.389942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8624634-7fe8-4e6e-8d79-db34fe9ea437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed404e2262d44a3294d19ca3dd263134, user_id=c13aa0bc422d4e8081bc704b437f5f52, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:46.392673 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8624634-7fe8-4e6e-8d79-db34fe9ea437 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:46.405801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1173/2330] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:09:46 2022] PUT /identity/v3/projects/ed404e2262d44a3294d19ca3dd263134/users/c13aa0bc422d4e8081bc704b437f5f52/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:09:46.409661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d751f-72ed-4a4b-974b-c40588f83028 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:46.420072 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d751f-72ed-4a4b-974b-c40588f83028 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:46.475885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc33db-f0a2-4191-b108-720d7ba13cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:46.476542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc33db-f0a2-4191-b108-720d7ba13cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:46.476821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc33db-f0a2-4191-b108-720d7ba13cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:46.477096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc33db-f0a2-4191-b108-720d7ba13cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:46.477812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc33db-f0a2-4191-b108-720d7ba13cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:46.479455 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc33db-f0a2-4191-b108-720d7ba13cf9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:46.489087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1158/2331] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:46.493462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8624634-7fe8-4e6e-8d79-db34fe9ea437 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:46.503475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8624634-7fe8-4e6e-8d79-db34fe9ea437 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:46.508498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6cf6a-c6e9-49de-8c5b-dfa1f9570130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:46.509368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6cf6a-c6e9-49de-8c5b-dfa1f9570130 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:46.509653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6cf6a-c6e9-49de-8c5b-dfa1f9570130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:46.509930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6cf6a-c6e9-49de-8c5b-dfa1f9570130 admin admin] PATH_INFO: `/v3/projects/ed404e2262d44a3294d19ca3dd263134/users/c13aa0bc422d4e8081bc704b437f5f52/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:46.511913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6cf6a-c6e9-49de-8c5b-dfa1f9570130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed404e2262d44a3294d19ca3dd263134, user_id=c13aa0bc422d4e8081bc704b437f5f52, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:46.514593 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b6cf6a-c6e9-49de-8c5b-dfa1f9570130 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:46.525184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1174/2332] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:09:46 2022] PUT /identity/v3/projects/ed404e2262d44a3294d19ca3dd263134/users/c13aa0bc422d4e8081bc704b437f5f52/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:46.537376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91afd8f-d29b-42d8-8c09-9d2e5362953b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:46.537670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91afd8f-d29b-42d8-8c09-9d2e5362953b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:46.537963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91afd8f-d29b-42d8-8c09-9d2e5362953b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:46.565161 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b91afd8f-d29b-42d8-8c09-9d2e5362953b None None] MFA Rules not processed for user `c13aa0bc422d4e8081bc704b437f5f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:46.566299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b91afd8f-d29b-42d8-8c09-9d2e5362953b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:46.623957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:46.634798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1159/2333] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:46 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:46.646433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6cf6a-c6e9-49de-8c5b-dfa1f9570130 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:46.648155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-70b6cf6a-c6e9-49de-8c5b-dfa1f9570130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:46.705959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6cf6a-c6e9-49de-8c5b-dfa1f9570130 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:46.710184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80152400-bda7-437a-9c0e-2652820fe94a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:46.710857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80152400-bda7-437a-9c0e-2652820fe94a service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:46.711146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80152400-bda7-437a-9c0e-2652820fe94a service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:46.711426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80152400-bda7-437a-9c0e-2652820fe94a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:46.720632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80152400-bda7-437a-9c0e-2652820fe94a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:46.722095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80152400-bda7-437a-9c0e-2652820fe94a service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:46.731722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1175/2334] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:46 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:46.804969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91afd8f-d29b-42d8-8c09-9d2e5362953b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:46.815067 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91afd8f-d29b-42d8-8c09-9d2e5362953b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:46.819725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558056e2-ab71-4190-a2fb-5feedff2c6d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:46.820368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558056e2-ab71-4190-a2fb-5feedff2c6d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:46.820649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558056e2-ab71-4190-a2fb-5feedff2c6d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:46.820924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558056e2-ab71-4190-a2fb-5feedff2c6d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:46.883150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558056e2-ab71-4190-a2fb-5feedff2c6d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:46.884606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558056e2-ab71-4190-a2fb-5feedff2c6d4 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:46.893471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1160/2335] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:46 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:49.587090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80152400-bda7-437a-9c0e-2652820fe94a service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:49.648130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80152400-bda7-437a-9c0e-2652820fe94a service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:49.652633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:49.653517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:49.653795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:49.654068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] PATH_INFO: `/v3/users/2f8aec4480264b8da80ae7c9a49e0a9f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:49.655276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8aec4480264b8da80ae7c9a49e0a9f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:49.657183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:49.700701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8aec4480264b8da80ae7c9a49e0a9f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:49.709109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:49.714143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8aec4480264b8da80ae7c9a49e0a9f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:49.721458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8aec4480264b8da80ae7c9a49e0a9f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:49.722112 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8aec4480264b8da80ae7c9a49e0a9f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:49.728978 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8aec4480264b8da80ae7c9a49e0a9f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:49.734844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1176/2336] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:09:49 2022] DELETE /identity/v3/users/2f8aec4480264b8da80ae7c9a49e0a9f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:49.739370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca1420-97b7-4b6f-9ac0-2378ab7f04ec None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:49.739836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca1420-97b7-4b6f-9ac0-2378ab7f04ec None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:49.739937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca1420-97b7-4b6f-9ac0-2378ab7f04ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:49.757640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cdca1420-97b7-4b6f-9ac0-2378ab7f04ec None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:49.758929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cdca1420-97b7-4b6f-9ac0-2378ab7f04ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:49.816659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1161/2337] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:49.826862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:49.828251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:49.888329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213e26c-893b-4c7a-afb4-bd81ff6a5715 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:49.891973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c6c59-e9d1-49c8-ba34-0b1d997e0285 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:49.892622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c6c59-e9d1-49c8-ba34-0b1d997e0285 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:49.892911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c6c59-e9d1-49c8-ba34-0b1d997e0285 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:49.893198 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c6c59-e9d1-49c8-ba34-0b1d997e0285 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:49.902640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c6c59-e9d1-49c8-ba34-0b1d997e0285 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:49.904072 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c6c59-e9d1-49c8-ba34-0b1d997e0285 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:49.913466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1177/2338] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:50.175765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca1420-97b7-4b6f-9ac0-2378ab7f04ec None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:50.178396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cdca1420-97b7-4b6f-9ac0-2378ab7f04ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:50.190125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca1420-97b7-4b6f-9ac0-2378ab7f04ec None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:50.195935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:50.197063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:50.197217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:50.197496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] PATH_INFO: `/v3/projects/1f272deab70b40d49f1d6ca517458be6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:50.198761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f272deab70b40d49f1d6ca517458be6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:50.200705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:50.249509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f272deab70b40d49f1d6ca517458be6', 'request_id': 'req-c2139981-481b-434d-a969-cb97f298fc69'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.255537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f272deab70b40d49f1d6ca517458be6', 'request_id': 'req-c2139981-481b-434d-a969-cb97f298fc69'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.262296 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f272deab70b40d49f1d6ca517458be6', 'request_id': 'req-c2139981-481b-434d-a969-cb97f298fc69'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.274840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] The token cache is being invalidate because project 1f272deab70b40d49f1d6ca517458be6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:09:50.275206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.275847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.277713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1162/2339] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:09:50 2022] DELETE /identity/v3/projects/1f272deab70b40d49f1d6ca517458be6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:50.281464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c6c59-e9d1-49c8-ba34-0b1d997e0285 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:50.291544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c6c59-e9d1-49c8-ba34-0b1d997e0285 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:50.346591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:50.347472 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:50.347757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:50.348565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] PATH_INFO: `/v3/users/c13aa0bc422d4e8081bc704b437f5f52` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:50.349854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13aa0bc422d4e8081bc704b437f5f52)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:50.351815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:50.397787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13aa0bc422d4e8081bc704b437f5f52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.409613 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13aa0bc422d4e8081bc704b437f5f52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.417178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13aa0bc422d4e8081bc704b437f5f52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.417805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13aa0bc422d4e8081bc704b437f5f52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.424668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-db6dc325-809b-4189-a2cf-390ff0aaf047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13aa0bc422d4e8081bc704b437f5f52'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.426060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:50.433090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1178/2340] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:09:50 2022] DELETE /identity/v3/users/c13aa0bc422d4e8081bc704b437f5f52 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:50.701864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:50.703343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:50.716775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2139981-481b-434d-a969-cb97f298fc69 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:50.729975 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:50.774597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:50.775495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:50.775723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:50.775999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] PATH_INFO: `/v3/projects/ed404e2262d44a3294d19ca3dd263134` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:50.777172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed404e2262d44a3294d19ca3dd263134)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:50.779123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:50.826063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed404e2262d44a3294d19ca3dd263134', 'request_id': 'req-a1918011-3ab3-4298-9651-06a897d74ba4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.831337 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed404e2262d44a3294d19ca3dd263134', 'request_id': 'req-a1918011-3ab3-4298-9651-06a897d74ba4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.837873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed404e2262d44a3294d19ca3dd263134', 'request_id': 'req-a1918011-3ab3-4298-9651-06a897d74ba4'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.850632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] The token cache is being invalidate because project ed404e2262d44a3294d19ca3dd263134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:09:50.850903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.851516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1918011-3ab3-4298-9651-06a897d74ba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:50.853039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1163/2341] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:09:50 2022] DELETE /identity/v3/projects/ed404e2262d44a3294d19ca3dd263134 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:50.858188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fd58d-24ae-44d3-8445-9b693c973081 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:50.858706 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fd58d-24ae-44d3-8445-9b693c973081 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:50.858804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fd58d-24ae-44d3-8445-9b693c973081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:50.876210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b64fd58d-24ae-44d3-8445-9b693c973081 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:50.877396 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b64fd58d-24ae-44d3-8445-9b693c973081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:50.934734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1179/2342] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:09:50.942283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754f19-3657-41b1-bb38-b48b6b343732 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:50.942609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754f19-3657-41b1-bb38-b48b6b343732 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:50.942894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754f19-3657-41b1-bb38-b48b6b343732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:50.961310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d3754f19-3657-41b1-bb38-b48b6b343732 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:50.962552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d3754f19-3657-41b1-bb38-b48b6b343732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:50.970538 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1164/2343] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:50.975367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fd58d-24ae-44d3-8445-9b693c973081 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:50.985029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fd58d-24ae-44d3-8445-9b693c973081 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:50.989631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ec4d-56b8-4882-aa4f-41360ea3f64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:50.990274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ec4d-56b8-4882-aa4f-41360ea3f64f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:50.990594 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ec4d-56b8-4882-aa4f-41360ea3f64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:50.990870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00ec4d-56b8-4882-aa4f-41360ea3f64f admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:50.991582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00ec4d-56b8-4882-aa4f-41360ea3f64f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:50.991886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00ec4d-56b8-4882-aa4f-41360ea3f64f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:50.993729 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00ec4d-56b8-4882-aa4f-41360ea3f64f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.005505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1180/2344] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:51.009725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3754f19-3657-41b1-bb38-b48b6b343732 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.018952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3754f19-3657-41b1-bb38-b48b6b343732 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.024184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197e187-7ed9-464f-8279-41ce4fc3801c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.024822 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197e187-7ed9-464f-8279-41ce4fc3801c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.025107 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197e187-7ed9-464f-8279-41ce4fc3801c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.025402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197e187-7ed9-464f-8279-41ce4fc3801c admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.026146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197e187-7ed9-464f-8279-41ce4fc3801c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.028061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197e187-7ed9-464f-8279-41ce4fc3801c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.052517 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1165/2345] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:09:51 2022] POST /identity/v3/projects => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:51.056662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ec4d-56b8-4882-aa4f-41360ea3f64f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.066635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00ec4d-56b8-4882-aa4f-41360ea3f64f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.121144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec89516-7459-4a6d-b9c2-3925adba6db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.121796 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec89516-7459-4a6d-b9c2-3925adba6db8 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.122081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec89516-7459-4a6d-b9c2-3925adba6db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.122357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec89516-7459-4a6d-b9c2-3925adba6db8 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.123202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec89516-7459-4a6d-b9c2-3925adba6db8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.125103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec89516-7459-4a6d-b9c2-3925adba6db8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.145886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1181/2346] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:09:51 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:51.150001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197e187-7ed9-464f-8279-41ce4fc3801c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.160384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197e187-7ed9-464f-8279-41ce4fc3801c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.165020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b25120-5771-4177-b9a2-c7ed4290ac25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.165667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25120-5771-4177-b9a2-c7ed4290ac25 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.165952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25120-5771-4177-b9a2-c7ed4290ac25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.166231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b25120-5771-4177-b9a2-c7ed4290ac25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.166988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b25120-5771-4177-b9a2-c7ed4290ac25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.168599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b25120-5771-4177-b9a2-c7ed4290ac25 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.178775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1166/2347] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:51.183756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec89516-7459-4a6d-b9c2-3925adba6db8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.195141 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec89516-7459-4a6d-b9c2-3925adba6db8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.199791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d109f98-73c7-4610-8c99-6656ca771c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.200450 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d109f98-73c7-4610-8c99-6656ca771c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.200732 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d109f98-73c7-4610-8c99-6656ca771c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.201035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d109f98-73c7-4610-8c99-6656ca771c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.201759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d109f98-73c7-4610-8c99-6656ca771c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.203390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d109f98-73c7-4610-8c99-6656ca771c84 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.214359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1182/2348] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:51.218502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b25120-5771-4177-b9a2-c7ed4290ac25 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.228495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b25120-5771-4177-b9a2-c7ed4290ac25 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.233143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9b2c3-1a23-496a-b643-18bb3334df97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.234012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9b2c3-1a23-496a-b643-18bb3334df97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.234296 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9b2c3-1a23-496a-b643-18bb3334df97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.234620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9b2c3-1a23-496a-b643-18bb3334df97 admin admin] PATH_INFO: `/v3/projects/cb651007d2e146e2bc71924737f80198/users/37d9b60a3cea4125a70e1877e5b29029/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.249203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9b2c3-1a23-496a-b643-18bb3334df97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb651007d2e146e2bc71924737f80198, user_id=37d9b60a3cea4125a70e1877e5b29029, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.251923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9b2c3-1a23-496a-b643-18bb3334df97 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.265313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1167/2349] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:09:51 2022] PUT /identity/v3/projects/cb651007d2e146e2bc71924737f80198/users/37d9b60a3cea4125a70e1877e5b29029/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:51.276035 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af82fa-e5af-45f6-8236-416d16a14a72 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.276377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af82fa-e5af-45f6-8236-416d16a14a72 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.276609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af82fa-e5af-45f6-8236-416d16a14a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.302220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-11af82fa-e5af-45f6-8236-416d16a14a72 None None] MFA Rules not processed for user `37d9b60a3cea4125a70e1877e5b29029`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:51.303432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-11af82fa-e5af-45f6-8236-416d16a14a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:51.355430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:51.367439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1183/2350] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:51.372397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9b2c3-1a23-496a-b643-18bb3334df97 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.417862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:51.436420 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9b2c3-1a23-496a-b643-18bb3334df97 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.441063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977139fb-5f3e-4ea0-b903-45486afe8e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.441783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977139fb-5f3e-4ea0-b903-45486afe8e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.442617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977139fb-5f3e-4ea0-b903-45486afe8e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.442895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977139fb-5f3e-4ea0-b903-45486afe8e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.443590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977139fb-5f3e-4ea0-b903-45486afe8e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.445524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977139fb-5f3e-4ea0-b903-45486afe8e4e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.474122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1168/2351] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:09:51 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:51.475297 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af82fa-e5af-45f6-8236-416d16a14a72 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.486003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af82fa-e5af-45f6-8236-416d16a14a72 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.547793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bec4d-975f-407c-ae19-c1455534c4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.548470 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bec4d-975f-407c-ae19-c1455534c4ae admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.548755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bec4d-975f-407c-ae19-c1455534c4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.549032 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bec4d-975f-407c-ae19-c1455534c4ae admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.549844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bec4d-975f-407c-ae19-c1455534c4ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.553219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bec4d-975f-407c-ae19-c1455534c4ae admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.576580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1184/2352] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:09:51 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:09:51.580791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977139fb-5f3e-4ea0-b903-45486afe8e4e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.591435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977139fb-5f3e-4ea0-b903-45486afe8e4e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.596121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab46c3-34fd-475b-8c32-c6bb93e3ca68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.596774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab46c3-34fd-475b-8c32-c6bb93e3ca68 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.597059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab46c3-34fd-475b-8c32-c6bb93e3ca68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.597335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab46c3-34fd-475b-8c32-c6bb93e3ca68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.598083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab46c3-34fd-475b-8c32-c6bb93e3ca68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.599733 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab46c3-34fd-475b-8c32-c6bb93e3ca68 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.610121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1169/2353] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:51.614755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bec4d-975f-407c-ae19-c1455534c4ae admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.624737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bec4d-975f-407c-ae19-c1455534c4ae admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.629936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe49f5-b46d-4de2-a594-614559e78c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.630865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe49f5-b46d-4de2-a594-614559e78c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.631147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe49f5-b46d-4de2-a594-614559e78c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.631422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe49f5-b46d-4de2-a594-614559e78c32 admin admin] PATH_INFO: `/v3/projects/76f07f3b8a7f4d9e8d49370e9b9243ff/users/0dc48ed44feb46ed9fe5f37b389390c2/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.646330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe49f5-b46d-4de2-a594-614559e78c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f07f3b8a7f4d9e8d49370e9b9243ff, user_id=0dc48ed44feb46ed9fe5f37b389390c2, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.649024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe49f5-b46d-4de2-a594-614559e78c32 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.661845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1185/2354] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:09:51 2022] PUT /identity/v3/projects/76f07f3b8a7f4d9e8d49370e9b9243ff/users/0dc48ed44feb46ed9fe5f37b389390c2/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:51.665720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab46c3-34fd-475b-8c32-c6bb93e3ca68 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.676738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab46c3-34fd-475b-8c32-c6bb93e3ca68 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.731576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2634c14-dc81-487d-9136-2a57f19897d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.732232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2634c14-dc81-487d-9136-2a57f19897d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.732515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2634c14-dc81-487d-9136-2a57f19897d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.732791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2634c14-dc81-487d-9136-2a57f19897d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.733542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2634c14-dc81-487d-9136-2a57f19897d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.735197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2634c14-dc81-487d-9136-2a57f19897d6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.746008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1170/2355] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:51.750144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe49f5-b46d-4de2-a594-614559e78c32 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.760294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe49f5-b46d-4de2-a594-614559e78c32 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.764847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48842f97-1376-4b0b-800a-de83cdb9ba3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.765716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48842f97-1376-4b0b-800a-de83cdb9ba3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.765998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48842f97-1376-4b0b-800a-de83cdb9ba3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.766274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48842f97-1376-4b0b-800a-de83cdb9ba3d admin admin] PATH_INFO: `/v3/projects/76f07f3b8a7f4d9e8d49370e9b9243ff/users/0dc48ed44feb46ed9fe5f37b389390c2/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.768252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48842f97-1376-4b0b-800a-de83cdb9ba3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f07f3b8a7f4d9e8d49370e9b9243ff, user_id=0dc48ed44feb46ed9fe5f37b389390c2, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.770916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48842f97-1376-4b0b-800a-de83cdb9ba3d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.780909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1186/2356] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:09:51 2022] PUT /identity/v3/projects/76f07f3b8a7f4d9e8d49370e9b9243ff/users/0dc48ed44feb46ed9fe5f37b389390c2/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:09:51.792607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b00cc0-7190-4d96-bc63-8881629a06e7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.793082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b00cc0-7190-4d96-bc63-8881629a06e7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.793187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b00cc0-7190-4d96-bc63-8881629a06e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.820492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d9b00cc0-7190-4d96-bc63-8881629a06e7 None None] MFA Rules not processed for user `0dc48ed44feb46ed9fe5f37b389390c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:51.821714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d9b00cc0-7190-4d96-bc63-8881629a06e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:51.895099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1171/2357] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:51.906340 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48842f97-1376-4b0b-800a-de83cdb9ba3d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:51.908100 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-48842f97-1376-4b0b-800a-de83cdb9ba3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:51.964812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48842f97-1376-4b0b-800a-de83cdb9ba3d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:51.968698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8050483-5c12-4f0a-bcb0-210c19fc9bee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:51.969344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8050483-5c12-4f0a-bcb0-210c19fc9bee service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:51.969633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8050483-5c12-4f0a-bcb0-210c19fc9bee service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:51.969913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8050483-5c12-4f0a-bcb0-210c19fc9bee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:51.979305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8050483-5c12-4f0a-bcb0-210c19fc9bee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:51.980730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8050483-5c12-4f0a-bcb0-210c19fc9bee service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:51.990714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1187/2358] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:52.034653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b00cc0-7190-4d96-bc63-8881629a06e7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:52.036509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d9b00cc0-7190-4d96-bc63-8881629a06e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:52.048193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b00cc0-7190-4d96-bc63-8881629a06e7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:52.052374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d66fc4-e754-4914-ba20-fc23be317726 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:52.053003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d66fc4-e754-4914-ba20-fc23be317726 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:52.053286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d66fc4-e754-4914-ba20-fc23be317726 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:52.053562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d66fc4-e754-4914-ba20-fc23be317726 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:52.090219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:52.110885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d66fc4-e754-4914-ba20-fc23be317726 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:52.112365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d66fc4-e754-4914-ba20-fc23be317726 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:52.122143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1172/2359] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:55.376968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8050483-5c12-4f0a-bcb0-210c19fc9bee service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:55.424930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:55.439097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8050483-5c12-4f0a-bcb0-210c19fc9bee service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:55.443607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:55.444500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:55.444785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:55.445064 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] PATH_INFO: `/v3/users/37d9b60a3cea4125a70e1877e5b29029` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:55.446243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d9b60a3cea4125a70e1877e5b29029)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:55.448246 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:55.494149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d9b60a3cea4125a70e1877e5b29029'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:55.506182 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d9b60a3cea4125a70e1877e5b29029'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:55.513672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d9b60a3cea4125a70e1877e5b29029'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:55.514363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d9b60a3cea4125a70e1877e5b29029'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:55.522295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d9b60a3cea4125a70e1877e5b29029'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:55.528389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1188/2360] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:09:55 2022] DELETE /identity/v3/users/37d9b60a3cea4125a70e1877e5b29029 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:55.532957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240360a-c40c-447a-ad8e-e83d4e234f26 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:55.533430 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240360a-c40c-447a-ad8e-e83d4e234f26 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:55.533540 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240360a-c40c-447a-ad8e-e83d4e234f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:55.551832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6240360a-c40c-447a-ad8e-e83d4e234f26 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:55.553300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6240360a-c40c-447a-ad8e-e83d4e234f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:55.610615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1173/2361] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:55.620290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:55.621780 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:55.679971 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fdf31-f065-4ab7-b2f6-9e2a0dff78e8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:55.683612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200d1e1-c4ca-47b2-b060-0324eddddcb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:55.684257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200d1e1-c4ca-47b2-b060-0324eddddcb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:55.684540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200d1e1-c4ca-47b2-b060-0324eddddcb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:55.684814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3200d1e1-c4ca-47b2-b060-0324eddddcb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:55.694099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200d1e1-c4ca-47b2-b060-0324eddddcb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:55.695561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3200d1e1-c4ca-47b2-b060-0324eddddcb8 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:55.704777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1189/2362] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:55.964022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240360a-c40c-447a-ad8e-e83d4e234f26 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:55.965517 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6240360a-c40c-447a-ad8e-e83d4e234f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:55.977701 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240360a-c40c-447a-ad8e-e83d4e234f26 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:55.982606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:55.983469 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:55.983750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:55.984025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] PATH_INFO: `/v3/projects/cb651007d2e146e2bc71924737f80198` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:55.986024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb651007d2e146e2bc71924737f80198)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:55.988214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:56.034658 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb651007d2e146e2bc71924737f80198', 'request_id': 'req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.040204 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb651007d2e146e2bc71924737f80198', 'request_id': 'req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.046410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb651007d2e146e2bc71924737f80198', 'request_id': 'req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.059306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] The token cache is being invalidate because project cb651007d2e146e2bc71924737f80198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:09:56.059592 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.060307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.061850 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1174/2363] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:09:55 2022] DELETE /identity/v3/projects/cb651007d2e146e2bc71924737f80198 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:56.065964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200d1e1-c4ca-47b2-b060-0324eddddcb8 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:56.076164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3200d1e1-c4ca-47b2-b060-0324eddddcb8 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:56.133946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:56.134858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:56.135144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:56.135416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] PATH_INFO: `/v3/users/0dc48ed44feb46ed9fe5f37b389390c2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:56.136597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dc48ed44feb46ed9fe5f37b389390c2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:56.138711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:56.151740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:56.188958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dc48ed44feb46ed9fe5f37b389390c2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.201941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dc48ed44feb46ed9fe5f37b389390c2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.209959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dc48ed44feb46ed9fe5f37b389390c2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.210714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dc48ed44feb46ed9fe5f37b389390c2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.217959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-e0dbc2b7-4a3a-422f-bdcf-db1f81224cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dc48ed44feb46ed9fe5f37b389390c2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.224790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1190/2364] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:09:56 2022] DELETE /identity/v3/users/0dc48ed44feb46ed9fe5f37b389390c2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:09:56.489046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:56.490556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:56.503923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4c7fc-e1b3-4a67-9f80-7a4645653a82 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:56.559699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:56.560581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:56.560863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:56.561136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] PATH_INFO: `/v3/projects/76f07f3b8a7f4d9e8d49370e9b9243ff` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:56.562313 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=76f07f3b8a7f4d9e8d49370e9b9243ff)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:56.564312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:56.566867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:56.614180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76f07f3b8a7f4d9e8d49370e9b9243ff', 'request_id': 'req-81d76dd5-fe07-439c-b691-8d3211b5c4ce'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.620321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76f07f3b8a7f4d9e8d49370e9b9243ff', 'request_id': 'req-81d76dd5-fe07-439c-b691-8d3211b5c4ce'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.626381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76f07f3b8a7f4d9e8d49370e9b9243ff', 'request_id': 'req-81d76dd5-fe07-439c-b691-8d3211b5c4ce'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.638322 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] The token cache is being invalidate because project 76f07f3b8a7f4d9e8d49370e9b9243ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:09:56.638640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.639255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-81d76dd5-fe07-439c-b691-8d3211b5c4ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:09:56.640782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1175/2365] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:09:56 2022] DELETE /identity/v3/projects/76f07f3b8a7f4d9e8d49370e9b9243ff => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:56.646225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01f7d3-deea-45a3-aec2-f6f668c9ad0d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:56.646701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01f7d3-deea-45a3-aec2-f6f668c9ad0d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:56.646962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f01f7d3-deea-45a3-aec2-f6f668c9ad0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:56.665018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1f01f7d3-deea-45a3-aec2-f6f668c9ad0d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:56.666157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1f01f7d3-deea-45a3-aec2-f6f668c9ad0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:56.726451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1191/2366] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:56.734153 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557909e-f88f-4dc2-bf76-f0b8ee713e9f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:56.734468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557909e-f88f-4dc2-bf76-f0b8ee713e9f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:56.734751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557909e-f88f-4dc2-bf76-f0b8ee713e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:56.752897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d557909e-f88f-4dc2-bf76-f0b8ee713e9f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:56.754086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d557909e-f88f-4dc2-bf76-f0b8ee713e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:56.761853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1176/2367] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:56.765819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01f7d3-deea-45a3-aec2-f6f668c9ad0d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:56.775906 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f01f7d3-deea-45a3-aec2-f6f668c9ad0d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:56.780517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60670626-a108-420d-926e-b7a4dcbb6948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:56.781168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60670626-a108-420d-926e-b7a4dcbb6948 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:56.781455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60670626-a108-420d-926e-b7a4dcbb6948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:56.781736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60670626-a108-420d-926e-b7a4dcbb6948 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:56.782515 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60670626-a108-420d-926e-b7a4dcbb6948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:56.782857 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60670626-a108-420d-926e-b7a4dcbb6948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:56.784451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60670626-a108-420d-926e-b7a4dcbb6948 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:56.796370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1192/2368] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:56.800545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557909e-f88f-4dc2-bf76-f0b8ee713e9f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:56.809740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557909e-f88f-4dc2-bf76-f0b8ee713e9f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:56.814329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f123f1c-a576-4876-8d37-d0b91c9ef721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:56.815044 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f123f1c-a576-4876-8d37-d0b91c9ef721 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:56.815270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f123f1c-a576-4876-8d37-d0b91c9ef721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:56.815542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f123f1c-a576-4876-8d37-d0b91c9ef721 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:56.816237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f123f1c-a576-4876-8d37-d0b91c9ef721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:56.818266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f123f1c-a576-4876-8d37-d0b91c9ef721 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:56.843546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1177/2369] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:09:56 2022] POST /identity/v3/projects => generated 393 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:56.847574 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60670626-a108-420d-926e-b7a4dcbb6948 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:56.857922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60670626-a108-420d-926e-b7a4dcbb6948 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:56.912512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe976c-ff0a-4b8c-b9a1-04ca39f05c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:56.913180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe976c-ff0a-4b8c-b9a1-04ca39f05c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:56.913467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe976c-ff0a-4b8c-b9a1-04ca39f05c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:56.913749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe976c-ff0a-4b8c-b9a1-04ca39f05c05 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:56.914433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe976c-ff0a-4b8c-b9a1-04ca39f05c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:56.916432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe976c-ff0a-4b8c-b9a1-04ca39f05c05 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:56.935295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1193/2370] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:09:56 2022] POST /identity/v3/users => generated 297 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:56.939371 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f123f1c-a576-4876-8d37-d0b91c9ef721 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:56.949542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f123f1c-a576-4876-8d37-d0b91c9ef721 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:56.954166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82771ffd-5bd2-485a-9fa6-148aa1523f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:56.954853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82771ffd-5bd2-485a-9fa6-148aa1523f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:56.955138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82771ffd-5bd2-485a-9fa6-148aa1523f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:56.955415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82771ffd-5bd2-485a-9fa6-148aa1523f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:56.956133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82771ffd-5bd2-485a-9fa6-148aa1523f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:56.958348 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82771ffd-5bd2-485a-9fa6-148aa1523f05 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:56.968897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1178/2371] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:56.973371 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe976c-ff0a-4b8c-b9a1-04ca39f05c05 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:56.983390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe976c-ff0a-4b8c-b9a1-04ca39f05c05 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:56.987994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9279c813-808a-4b86-8fc8-84606f86d26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:56.988640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c813-808a-4b86-8fc8-84606f86d26e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:56.988929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c813-808a-4b86-8fc8-84606f86d26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:56.989211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9279c813-808a-4b86-8fc8-84606f86d26e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:56.989931 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279c813-808a-4b86-8fc8-84606f86d26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:56.991583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9279c813-808a-4b86-8fc8-84606f86d26e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:56.993055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:57.005357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1194/2372] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:57.008978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82771ffd-5bd2-485a-9fa6-148aa1523f05 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:57.018775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82771ffd-5bd2-485a-9fa6-148aa1523f05 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:57.023977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621fb5a-56c7-4af1-9a5b-afd113b5a2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:57.024841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621fb5a-56c7-4af1-9a5b-afd113b5a2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.025120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621fb5a-56c7-4af1-9a5b-afd113b5a2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.025392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621fb5a-56c7-4af1-9a5b-afd113b5a2c2 admin admin] PATH_INFO: `/v3/projects/1072b42cb1ad4bc39ea74f1092b42dd6/users/dc7da5081696423d826bbee0f05d6685/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.039805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621fb5a-56c7-4af1-9a5b-afd113b5a2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1072b42cb1ad4bc39ea74f1092b42dd6, user_id=dc7da5081696423d826bbee0f05d6685, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:57.042532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621fb5a-56c7-4af1-9a5b-afd113b5a2c2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:57.055769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1179/2373] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:09:57 2022] PUT /identity/v3/projects/1072b42cb1ad4bc39ea74f1092b42dd6/users/dc7da5081696423d826bbee0f05d6685/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:57.067673 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c030c0-71d3-46a5-aaaf-eb7bf22f674f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.067987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c030c0-71d3-46a5-aaaf-eb7bf22f674f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.068253 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c030c0-71d3-46a5-aaaf-eb7bf22f674f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.094063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-50c030c0-71d3-46a5-aaaf-eb7bf22f674f None None] MFA Rules not processed for user `dc7da5081696423d826bbee0f05d6685`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:57.095294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-50c030c0-71d3-46a5-aaaf-eb7bf22f674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:57.157098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1195/2374] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:57.161609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621fb5a-56c7-4af1-9a5b-afd113b5a2c2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:57.226139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621fb5a-56c7-4af1-9a5b-afd113b5a2c2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:57.230810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89634da6-3c6b-45f9-869b-b5fedb47de26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:57.231466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634da6-3c6b-45f9-869b-b5fedb47de26 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.231752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634da6-3c6b-45f9-869b-b5fedb47de26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.232033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89634da6-3c6b-45f9-869b-b5fedb47de26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.232720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89634da6-3c6b-45f9-869b-b5fedb47de26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:57.234710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89634da6-3c6b-45f9-869b-b5fedb47de26 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:57.260512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1180/2375] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:09:57 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:09:57.264765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c030c0-71d3-46a5-aaaf-eb7bf22f674f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:57.276050 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c030c0-71d3-46a5-aaaf-eb7bf22f674f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:57.331765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d2049-2961-44dc-ae47-52db105eedfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:57.332992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d2049-2961-44dc-ae47-52db105eedfb admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.333275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d2049-2961-44dc-ae47-52db105eedfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.333549 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d2049-2961-44dc-ae47-52db105eedfb admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.334233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d2049-2961-44dc-ae47-52db105eedfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:57.336188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d2049-2961-44dc-ae47-52db105eedfb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:57.356521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1196/2376] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:09:57 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:09:57.360446 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89634da6-3c6b-45f9-869b-b5fedb47de26 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:57.371954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89634da6-3c6b-45f9-869b-b5fedb47de26 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:57.377021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16f7a9-65db-4779-b340-31ddd1f4671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:57.378345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16f7a9-65db-4779-b340-31ddd1f4671d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.378671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16f7a9-65db-4779-b340-31ddd1f4671d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.379584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16f7a9-65db-4779-b340-31ddd1f4671d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.380329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16f7a9-65db-4779-b340-31ddd1f4671d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:57.382023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da16f7a9-65db-4779-b340-31ddd1f4671d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:57.392884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1181/2377] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:57.397064 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d2049-2961-44dc-ae47-52db105eedfb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:57.407089 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d2049-2961-44dc-ae47-52db105eedfb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:57.411819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca13d62-da1c-4de5-b3e4-938995d2af91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:57.412787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca13d62-da1c-4de5-b3e4-938995d2af91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.413144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca13d62-da1c-4de5-b3e4-938995d2af91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.413492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca13d62-da1c-4de5-b3e4-938995d2af91 admin admin] PATH_INFO: `/v3/projects/932e24f1d2a24ad88080a98bfc70075a/users/78185d58315640219c99d7d106b7ffdd/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.428543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca13d62-da1c-4de5-b3e4-938995d2af91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=932e24f1d2a24ad88080a98bfc70075a, user_id=78185d58315640219c99d7d106b7ffdd, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:57.431251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca13d62-da1c-4de5-b3e4-938995d2af91 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:57.445561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1197/2378] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:09:57 2022] PUT /identity/v3/projects/932e24f1d2a24ad88080a98bfc70075a/users/78185d58315640219c99d7d106b7ffdd/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:57.449428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16f7a9-65db-4779-b340-31ddd1f4671d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:57.454614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:57.461155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da16f7a9-65db-4779-b340-31ddd1f4671d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:57.520102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac646e8e-4f19-49f8-ad33-b89b455650e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:57.520661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646e8e-4f19-49f8-ad33-b89b455650e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.520945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646e8e-4f19-49f8-ad33-b89b455650e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.521223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646e8e-4f19-49f8-ad33-b89b455650e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.521939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac646e8e-4f19-49f8-ad33-b89b455650e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:57.523604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac646e8e-4f19-49f8-ad33-b89b455650e2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:57.533129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1182/2379] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:09:57.537911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca13d62-da1c-4de5-b3e4-938995d2af91 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:57.548495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca13d62-da1c-4de5-b3e4-938995d2af91 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:57.553852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117f3c2-a0b1-4b7a-b87d-c44621af32ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:57.554778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117f3c2-a0b1-4b7a-b87d-c44621af32ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.555066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117f3c2-a0b1-4b7a-b87d-c44621af32ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.555346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5117f3c2-a0b1-4b7a-b87d-c44621af32ef admin admin] PATH_INFO: `/v3/projects/932e24f1d2a24ad88080a98bfc70075a/users/78185d58315640219c99d7d106b7ffdd/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.557320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117f3c2-a0b1-4b7a-b87d-c44621af32ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=932e24f1d2a24ad88080a98bfc70075a, user_id=78185d58315640219c99d7d106b7ffdd, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:57.560075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5117f3c2-a0b1-4b7a-b87d-c44621af32ef admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:57.571001 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1198/2380] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:09:57 2022] PUT /identity/v3/projects/932e24f1d2a24ad88080a98bfc70075a/users/78185d58315640219c99d7d106b7ffdd/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:57.581227 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf679a-1f3d-4cae-b69e-3a392f32d93b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.581699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf679a-1f3d-4cae-b69e-3a392f32d93b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.581794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf679a-1f3d-4cae-b69e-3a392f32d93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.607254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-79cf679a-1f3d-4cae-b69e-3a392f32d93b None None] MFA Rules not processed for user `78185d58315640219c99d7d106b7ffdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:57.608442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-79cf679a-1f3d-4cae-b69e-3a392f32d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:57.678591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1183/2381] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:09:57.689129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117f3c2-a0b1-4b7a-b87d-c44621af32ef admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:57.690952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5117f3c2-a0b1-4b7a-b87d-c44621af32ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:57.732661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:57.752372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5117f3c2-a0b1-4b7a-b87d-c44621af32ef admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:57.756030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018beeb-926b-4eb4-a2ce-43b034351af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:57.756672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018beeb-926b-4eb4-a2ce-43b034351af1 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.756958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018beeb-926b-4eb4-a2ce-43b034351af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.757240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018beeb-926b-4eb4-a2ce-43b034351af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.766070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018beeb-926b-4eb4-a2ce-43b034351af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:57.767543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018beeb-926b-4eb4-a2ce-43b034351af1 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:57.776879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1199/2382] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:57.821476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf679a-1f3d-4cae-b69e-3a392f32d93b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:57.822882 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-79cf679a-1f3d-4cae-b69e-3a392f32d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:57.834903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf679a-1f3d-4cae-b69e-3a392f32d93b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:57.838419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbf5b-024a-41b5-93b4-766c93327f86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:57.839070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbf5b-024a-41b5-93b4-766c93327f86 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:57.839352 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbf5b-024a-41b5-93b4-766c93327f86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:57.839626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbf5b-024a-41b5-93b4-766c93327f86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:57.895721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fbf5b-024a-41b5-93b4-766c93327f86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:57.897203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fbf5b-024a-41b5-93b4-766c93327f86 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:57.906096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1184/2383] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:09:58.180529 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018beeb-926b-4eb4-a2ce-43b034351af1 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:58.182050 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9018beeb-926b-4eb4-a2ce-43b034351af1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:58.260106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018beeb-926b-4eb4-a2ce-43b034351af1 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:58.265528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b5619-93b3-4d97-a075-084d73cdbcb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:58.266203 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5619-93b3-4d97-a075-084d73cdbcb1 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:58.266516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5619-93b3-4d97-a075-084d73cdbcb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:58.266801 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5619-93b3-4d97-a075-084d73cdbcb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:58.268691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-de7b5619-93b3-4d97-a075-084d73cdbcb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:58.328013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b5619-93b3-4d97-a075-084d73cdbcb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:58.329509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b5619-93b3-4d97-a075-084d73cdbcb1 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:58.338879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1200/2384] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:09:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:01.190208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbf5b-024a-41b5-93b4-766c93327f86 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:01.221371 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:01.261500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbf5b-024a-41b5-93b4-766c93327f86 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:01.265961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:01.266849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:01.267128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:01.267404 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] PATH_INFO: `/v3/users/dc7da5081696423d826bbee0f05d6685` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:01.268574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc7da5081696423d826bbee0f05d6685)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:01.270568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:01.313187 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc7da5081696423d826bbee0f05d6685'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:01.313822 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc7da5081696423d826bbee0f05d6685'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:01.321142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc7da5081696423d826bbee0f05d6685'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:01.321740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc7da5081696423d826bbee0f05d6685'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:01.333223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc7da5081696423d826bbee0f05d6685'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:01.344935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1185/2385] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:10:01 2022] DELETE /identity/v3/users/dc7da5081696423d826bbee0f05d6685 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:01.349520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae0e4e-3de0-4ada-acd2-fd26e52d282c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:01.349994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae0e4e-3de0-4ada-acd2-fd26e52d282c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:01.350134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae0e4e-3de0-4ada-acd2-fd26e52d282c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:01.367705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-deae0e4e-3de0-4ada-acd2-fd26e52d282c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:01.368953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-deae0e4e-3de0-4ada-acd2-fd26e52d282c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:01.426370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1201/2386] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:10:01.435988 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:01.437358 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:01.493700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7f8f2-763d-4ac4-b9c9-9ed38fdd6867 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:01.497275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f487e93-6f03-41e8-9823-9f5b3948ff39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:01.497909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f487e93-6f03-41e8-9823-9f5b3948ff39 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:01.498190 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f487e93-6f03-41e8-9823-9f5b3948ff39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:01.498492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f487e93-6f03-41e8-9823-9f5b3948ff39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:01.507468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f487e93-6f03-41e8-9823-9f5b3948ff39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:01.508901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f487e93-6f03-41e8-9823-9f5b3948ff39 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:01.518149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1186/2387] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:01.808429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae0e4e-3de0-4ada-acd2-fd26e52d282c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:01.810053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-deae0e4e-3de0-4ada-acd2-fd26e52d282c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:01.823101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae0e4e-3de0-4ada-acd2-fd26e52d282c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:01.827825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:01.828758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:01.829107 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:01.829451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] PATH_INFO: `/v3/projects/1072b42cb1ad4bc39ea74f1092b42dd6` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:01.830787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1072b42cb1ad4bc39ea74f1092b42dd6)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:01.833570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:01.846079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:01.883440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1072b42cb1ad4bc39ea74f1092b42dd6', 'request_id': 'req-1e163974-8219-4bc5-8080-5a0724d5a617'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:01.889572 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1072b42cb1ad4bc39ea74f1092b42dd6', 'request_id': 'req-1e163974-8219-4bc5-8080-5a0724d5a617'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:01.898925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1072b42cb1ad4bc39ea74f1092b42dd6', 'request_id': 'req-1e163974-8219-4bc5-8080-5a0724d5a617'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:01.912535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] The token cache is being invalidate because project 1072b42cb1ad4bc39ea74f1092b42dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:10:01.912824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:01.913442 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:01.915013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1202/2388] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:10:01 2022] DELETE /identity/v3/projects/1072b42cb1ad4bc39ea74f1092b42dd6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:01.919084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f487e93-6f03-41e8-9823-9f5b3948ff39 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:01.930637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f487e93-6f03-41e8-9823-9f5b3948ff39 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:01.982009 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:02.001585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:02.002660 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:02.003040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:02.003386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] PATH_INFO: `/v3/users/78185d58315640219c99d7d106b7ffdd` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:02.004729 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78185d58315640219c99d7d106b7ffdd)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:02.006904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:02.073760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78185d58315640219c99d7d106b7ffdd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:02.073760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78185d58315640219c99d7d106b7ffdd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:02.088771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78185d58315640219c99d7d106b7ffdd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:02.089029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78185d58315640219c99d7d106b7ffdd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:02.111514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1f7e2cc8-c03a-4094-abaa-baf4817b2f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78185d58315640219c99d7d106b7ffdd'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:02.126526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1187/2389] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:10:01 2022] DELETE /identity/v3/users/78185d58315640219c99d7d106b7ffdd => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:10:02.459227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:02.460731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:02.481405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e163974-8219-4bc5-8080-5a0724d5a617 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:02.556241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:02.557107 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:02.557387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:02.557660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] PATH_INFO: `/v3/projects/932e24f1d2a24ad88080a98bfc70075a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:02.558872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=932e24f1d2a24ad88080a98bfc70075a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:02.560785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:02.607550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932e24f1d2a24ad88080a98bfc70075a', 'request_id': 'req-2a4abfd3-040f-4732-a0da-2d23dd22bd59'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:02.614095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932e24f1d2a24ad88080a98bfc70075a', 'request_id': 'req-2a4abfd3-040f-4732-a0da-2d23dd22bd59'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:02.619842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932e24f1d2a24ad88080a98bfc70075a', 'request_id': 'req-2a4abfd3-040f-4732-a0da-2d23dd22bd59'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:02.632145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] The token cache is being invalidate because project 932e24f1d2a24ad88080a98bfc70075a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:10:02.632433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:02.633045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2a4abfd3-040f-4732-a0da-2d23dd22bd59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:02.634630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1203/2390] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:10:02 2022] DELETE /identity/v3/projects/932e24f1d2a24ad88080a98bfc70075a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:02.639459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bfa76-06e7-4838-966b-9bc15384a2eb None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:02.639909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bfa76-06e7-4838-966b-9bc15384a2eb None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:02.640018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bfa76-06e7-4838-966b-9bc15384a2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:02.658063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3d3bfa76-06e7-4838-966b-9bc15384a2eb None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:02.659291 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3d3bfa76-06e7-4838-966b-9bc15384a2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:02.719322 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1188/2391] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:10:02.727332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b4c08-7104-41b6-8f19-27faeaea1a2b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:02.727622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b4c08-7104-41b6-8f19-27faeaea1a2b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:02.727930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b4c08-7104-41b6-8f19-27faeaea1a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:02.746561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-959b4c08-7104-41b6-8f19-27faeaea1a2b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:02.747762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-959b4c08-7104-41b6-8f19-27faeaea1a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:02.756115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1204/2392] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:02.760230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bfa76-06e7-4838-966b-9bc15384a2eb None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:02.769806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bfa76-06e7-4838-966b-9bc15384a2eb None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:02.774375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaa5dd-66e4-44ad-ad6f-40b12b0ee751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:02.775061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaa5dd-66e4-44ad-ad6f-40b12b0ee751 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:02.775347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaa5dd-66e4-44ad-ad6f-40b12b0ee751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:02.775626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaa5dd-66e4-44ad-ad6f-40b12b0ee751 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:02.776351 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaa5dd-66e4-44ad-ad6f-40b12b0ee751 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:10:02.776661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaa5dd-66e4-44ad-ad6f-40b12b0ee751 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:02.778270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaa5dd-66e4-44ad-ad6f-40b12b0ee751 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:02.789640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1189/2393] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:10:02.793708 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b4c08-7104-41b6-8f19-27faeaea1a2b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:02.803684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b4c08-7104-41b6-8f19-27faeaea1a2b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:02.808312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88745ea5-09e8-4f53-97ad-720543691e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:02.809017 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88745ea5-09e8-4f53-97ad-720543691e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:02.809301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88745ea5-09e8-4f53-97ad-720543691e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:02.809582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88745ea5-09e8-4f53-97ad-720543691e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:02.810274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88745ea5-09e8-4f53-97ad-720543691e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:02.812220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88745ea5-09e8-4f53-97ad-720543691e4e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:02.838067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1205/2394] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:10:02 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:02.842296 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaa5dd-66e4-44ad-ad6f-40b12b0ee751 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:02.852343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaa5dd-66e4-44ad-ad6f-40b12b0ee751 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:02.898849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:02.909948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fca0f-09b5-4855-858f-4e59922b4d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:02.910640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fca0f-09b5-4855-858f-4e59922b4d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:02.910922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fca0f-09b5-4855-858f-4e59922b4d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:02.911199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84fca0f-09b5-4855-858f-4e59922b4d1b admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:02.911917 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84fca0f-09b5-4855-858f-4e59922b4d1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:02.913864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84fca0f-09b5-4855-858f-4e59922b4d1b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:02.933395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1190/2395] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:10:02 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:02.937343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88745ea5-09e8-4f53-97ad-720543691e4e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:02.943413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:02.949558 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88745ea5-09e8-4f53-97ad-720543691e4e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:02.954958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9adb0-b689-4726-8c03-f8a5e6dc21ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:02.955608 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9adb0-b689-4726-8c03-f8a5e6dc21ff admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:02.955890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9adb0-b689-4726-8c03-f8a5e6dc21ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:02.956165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9adb0-b689-4726-8c03-f8a5e6dc21ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:02.956879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9adb0-b689-4726-8c03-f8a5e6dc21ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:02.958492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9adb0-b689-4726-8c03-f8a5e6dc21ff admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:02.969463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1206/2396] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:02.974262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fca0f-09b5-4855-858f-4e59922b4d1b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:02.984347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84fca0f-09b5-4855-858f-4e59922b4d1b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:02.989534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ed98d-f186-4b2b-a9f8-82597aa9da4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:02.990180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ed98d-f186-4b2b-a9f8-82597aa9da4d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:02.990492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ed98d-f186-4b2b-a9f8-82597aa9da4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:02.990803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ed98d-f186-4b2b-a9f8-82597aa9da4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:02.991523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ed98d-f186-4b2b-a9f8-82597aa9da4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:02.993408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ed98d-f186-4b2b-a9f8-82597aa9da4d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:03.004897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1191/2397] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:03.008915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9adb0-b689-4726-8c03-f8a5e6dc21ff admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:03.018848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9adb0-b689-4726-8c03-f8a5e6dc21ff admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:03.023354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5a053-e856-4dbc-89c8-0077be90a15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:03.024220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a053-e856-4dbc-89c8-0077be90a15b admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.024501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a053-e856-4dbc-89c8-0077be90a15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.024776 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5a053-e856-4dbc-89c8-0077be90a15b admin admin] PATH_INFO: `/v3/projects/db17158c248d477fa4e884bc774c5799/users/baba5cc663684734b7bbe84c39ea63a7/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.039404 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5a053-e856-4dbc-89c8-0077be90a15b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db17158c248d477fa4e884bc774c5799, user_id=baba5cc663684734b7bbe84c39ea63a7, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:03.042056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5a053-e856-4dbc-89c8-0077be90a15b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:03.055912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1207/2398] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:10:03 2022] PUT /identity/v3/projects/db17158c248d477fa4e884bc774c5799/users/baba5cc663684734b7bbe84c39ea63a7/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:03.066884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e979409-db8e-4427-a11b-e1c67945e463 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.067197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e979409-db8e-4427-a11b-e1c67945e463 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.067481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e979409-db8e-4427-a11b-e1c67945e463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.093118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3e979409-db8e-4427-a11b-e1c67945e463 None None] MFA Rules not processed for user `baba5cc663684734b7bbe84c39ea63a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:03.094341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3e979409-db8e-4427-a11b-e1c67945e463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:03.155978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1192/2399] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:03.161019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5a053-e856-4dbc-89c8-0077be90a15b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:03.226236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5a053-e856-4dbc-89c8-0077be90a15b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:03.231243 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0956d3-684d-45ae-97d4-256ffa706a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:03.231910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0956d3-684d-45ae-97d4-256ffa706a76 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.232309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0956d3-684d-45ae-97d4-256ffa706a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.232602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0956d3-684d-45ae-97d4-256ffa706a76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.233306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0956d3-684d-45ae-97d4-256ffa706a76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:03.235261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0956d3-684d-45ae-97d4-256ffa706a76 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:03.271211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1208/2400] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:10:03 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:03.276924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e979409-db8e-4427-a11b-e1c67945e463 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:03.288490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e979409-db8e-4427-a11b-e1c67945e463 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:03.344843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd893ed-57a5-4e02-be86-9ac65ff16011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:03.345509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd893ed-57a5-4e02-be86-9ac65ff16011 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.345835 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd893ed-57a5-4e02-be86-9ac65ff16011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.346077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd893ed-57a5-4e02-be86-9ac65ff16011 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.346797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd893ed-57a5-4e02-be86-9ac65ff16011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:03.349342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd893ed-57a5-4e02-be86-9ac65ff16011 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:03.370257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1193/2401] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:10:03 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:03.374060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0956d3-684d-45ae-97d4-256ffa706a76 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:03.385086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0956d3-684d-45ae-97d4-256ffa706a76 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:03.389758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb766b-e5a5-4d75-b5a3-217479629d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:03.390419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb766b-e5a5-4d75-b5a3-217479629d03 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.390736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb766b-e5a5-4d75-b5a3-217479629d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.391011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb766b-e5a5-4d75-b5a3-217479629d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.391756 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb766b-e5a5-4d75-b5a3-217479629d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:03.393346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb766b-e5a5-4d75-b5a3-217479629d03 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:03.405321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1209/2402] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:03.409398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd893ed-57a5-4e02-be86-9ac65ff16011 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:03.419714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd893ed-57a5-4e02-be86-9ac65ff16011 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:03.425905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67900fc-e5a9-4052-9533-220645faaca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:03.426425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67900fc-e5a9-4052-9533-220645faaca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.426734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67900fc-e5a9-4052-9533-220645faaca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.427008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67900fc-e5a9-4052-9533-220645faaca0 admin admin] PATH_INFO: `/v3/projects/1e8a62ef72384bb4a8c351ec103ee3ce/users/0910f39a1d3e42bbba53e42f8406aa3d/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.444704 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67900fc-e5a9-4052-9533-220645faaca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8a62ef72384bb4a8c351ec103ee3ce, user_id=0910f39a1d3e42bbba53e42f8406aa3d, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:03.447458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67900fc-e5a9-4052-9533-220645faaca0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:03.462293 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1194/2403] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:10:03 2022] PUT /identity/v3/projects/1e8a62ef72384bb4a8c351ec103ee3ce/users/0910f39a1d3e42bbba53e42f8406aa3d/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:03.466420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb766b-e5a5-4d75-b5a3-217479629d03 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:03.477914 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb766b-e5a5-4d75-b5a3-217479629d03 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:03.537342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf95f5-6284-4bd5-bdb8-9fbbf05c2942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:03.538044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf95f5-6284-4bd5-bdb8-9fbbf05c2942 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.538407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf95f5-6284-4bd5-bdb8-9fbbf05c2942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.538740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf95f5-6284-4bd5-bdb8-9fbbf05c2942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.539465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf95f5-6284-4bd5-bdb8-9fbbf05c2942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:03.541074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf95f5-6284-4bd5-bdb8-9fbbf05c2942 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:03.555041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1210/2404] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:03.556752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67900fc-e5a9-4052-9533-220645faaca0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:03.567180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67900fc-e5a9-4052-9533-220645faaca0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:03.572032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161ec9e-45c1-4e90-9cd7-a1712719d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:03.572909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161ec9e-45c1-4e90-9cd7-a1712719d8e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.573196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161ec9e-45c1-4e90-9cd7-a1712719d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.573497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161ec9e-45c1-4e90-9cd7-a1712719d8e3 admin admin] PATH_INFO: `/v3/projects/1e8a62ef72384bb4a8c351ec103ee3ce/users/0910f39a1d3e42bbba53e42f8406aa3d/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.576071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161ec9e-45c1-4e90-9cd7-a1712719d8e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8a62ef72384bb4a8c351ec103ee3ce, user_id=0910f39a1d3e42bbba53e42f8406aa3d, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:03.578910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161ec9e-45c1-4e90-9cd7-a1712719d8e3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:03.592373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1195/2405] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:10:03 2022] PUT /identity/v3/projects/1e8a62ef72384bb4a8c351ec103ee3ce/users/0910f39a1d3e42bbba53e42f8406aa3d/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:10:03.603380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21b958-395a-4669-a226-fb3586ca9a0a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.603679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21b958-395a-4669-a226-fb3586ca9a0a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.603969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21b958-395a-4669-a226-fb3586ca9a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.631956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ce21b958-395a-4669-a226-fb3586ca9a0a None None] MFA Rules not processed for user `0910f39a1d3e42bbba53e42f8406aa3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:03.633191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ce21b958-395a-4669-a226-fb3586ca9a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:03.661431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:03.663882 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00adf68-79ac-4173-922b-37c1d8fcc329 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.664353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00adf68-79ac-4173-922b-37c1d8fcc329 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.664626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00adf68-79ac-4173-922b-37c1d8fcc329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.684677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f00adf68-79ac-4173-922b-37c1d8fcc329 None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:03.685890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f00adf68-79ac-4173-922b-37c1d8fcc329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:03.692526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:03.706228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1211/2406] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:03.717360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21b958-395a-4669-a226-fb3586ca9a0a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:03.718755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ce21b958-395a-4669-a226-fb3586ca9a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:03.744940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1196/2407] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:10:03.778538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21b958-395a-4669-a226-fb3586ca9a0a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:03.782348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498f665-33c9-44c8-b409-9229ec833804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:03.783038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498f665-33c9-44c8-b409-9229ec833804 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.783327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498f665-33c9-44c8-b409-9229ec833804 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.783610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498f665-33c9-44c8-b409-9229ec833804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.785859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00adf68-79ac-4173-922b-37c1d8fcc329 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:03.787251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f00adf68-79ac-4173-922b-37c1d8fcc329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:03.843795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2498f665-33c9-44c8-b409-9229ec833804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:03.845242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2498f665-33c9-44c8-b409-9229ec833804 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:03.855532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1212/2408] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:10:03.856772 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00adf68-79ac-4173-922b-37c1d8fcc329 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:03.861876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389b677-02f6-447c-b0c5-3e30a4ef6526 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:03.862464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389b677-02f6-447c-b0c5-3e30a4ef6526 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:03.862763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389b677-02f6-447c-b0c5-3e30a4ef6526 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:03.863045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389b677-02f6-447c-b0c5-3e30a4ef6526 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:03.871876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389b677-02f6-447c-b0c5-3e30a4ef6526 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:03.873575 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389b677-02f6-447c-b0c5-3e30a4ef6526 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:03.883106 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1197/2409] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:10:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 21:10:04.246815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498f665-33c9-44c8-b409-9229ec833804 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:04.248890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2498f665-33c9-44c8-b409-9229ec833804 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:04.426313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498f665-33c9-44c8-b409-9229ec833804 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:04.443927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eea550-7416-4e62-a6d3-1c70a8f04693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:04.446380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eea550-7416-4e62-a6d3-1c70a8f04693 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:04.447029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eea550-7416-4e62-a6d3-1c70a8f04693 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:04.447859 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eea550-7416-4e62-a6d3-1c70a8f04693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:04.450382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-87eea550-7416-4e62-a6d3-1c70a8f04693 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:04.468603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eea550-7416-4e62-a6d3-1c70a8f04693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:04.470364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eea550-7416-4e62-a6d3-1c70a8f04693 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:04.483940 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1213/2410] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:08.523203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389b677-02f6-447c-b0c5-3e30a4ef6526 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:08.531001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389b677-02f6-447c-b0c5-3e30a4ef6526 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:08.535331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc5046-4d56-424f-88b7-0ceb5fc0ea11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:08.536229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc5046-4d56-424f-88b7-0ceb5fc0ea11 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:08.536585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc5046-4d56-424f-88b7-0ceb5fc0ea11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:08.536941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc5046-4d56-424f-88b7-0ceb5fc0ea11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:08.549545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc5046-4d56-424f-88b7-0ceb5fc0ea11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:08.551253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bc5046-4d56-424f-88b7-0ceb5fc0ea11 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:08.562197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1198/2411] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:12.168417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eea550-7416-4e62-a6d3-1c70a8f04693 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:12.175335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-87eea550-7416-4e62-a6d3-1c70a8f04693 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:12.193161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:12.215850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eea550-7416-4e62-a6d3-1c70a8f04693 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:12.223412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3372d-fe97-4813-ac53-9c657a968da3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:12.224425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3372d-fe97-4813-ac53-9c657a968da3 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:12.224932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3372d-fe97-4813-ac53-9c657a968da3 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:12.225428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3372d-fe97-4813-ac53-9c657a968da3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:12.227653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-60c3372d-fe97-4813-ac53-9c657a968da3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:12.242918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3372d-fe97-4813-ac53-9c657a968da3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:12.244999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3372d-fe97-4813-ac53-9c657a968da3 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:12.261607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1214/2412] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:13.971092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc5046-4d56-424f-88b7-0ceb5fc0ea11 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:14.039207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc5046-4d56-424f-88b7-0ceb5fc0ea11 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:14.044079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:14.044578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:14.044858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:14.045133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] PATH_INFO: `/v3/users/baba5cc663684734b7bbe84c39ea63a7` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:14.046309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=baba5cc663684734b7bbe84c39ea63a7)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:14.048416 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:14.088148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:14.100879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baba5cc663684734b7bbe84c39ea63a7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.101711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baba5cc663684734b7bbe84c39ea63a7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.110524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baba5cc663684734b7bbe84c39ea63a7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.111111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baba5cc663684734b7bbe84c39ea63a7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.125807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baba5cc663684734b7bbe84c39ea63a7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.142270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1199/2413] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:10:13 2022] DELETE /identity/v3/users/baba5cc663684734b7bbe84c39ea63a7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:14.147411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b025a-ef53-47b3-a7d6-43e2939cdef7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:14.148117 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b025a-ef53-47b3-a7d6-43e2939cdef7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:14.148380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67b025a-ef53-47b3-a7d6-43e2939cdef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:14.169295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a67b025a-ef53-47b3-a7d6-43e2939cdef7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:14.170752 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a67b025a-ef53-47b3-a7d6-43e2939cdef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:14.247451 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1215/2414] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:14.261226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:14.262752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:14.344551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34222ec-627a-4bd7-9432-943bc0c20b3d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:14.349847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d01ae3-ffec-4feb-b1b7-e29a0f856655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:14.350597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d01ae3-ffec-4feb-b1b7-e29a0f856655 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:14.350888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d01ae3-ffec-4feb-b1b7-e29a0f856655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:14.351370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d01ae3-ffec-4feb-b1b7-e29a0f856655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:14.362317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d01ae3-ffec-4feb-b1b7-e29a0f856655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:14.364145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d01ae3-ffec-4feb-b1b7-e29a0f856655 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:14.375681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1200/2415] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:14.538390 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b025a-ef53-47b3-a7d6-43e2939cdef7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:14.551350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67b025a-ef53-47b3-a7d6-43e2939cdef7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:14.559347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48450-0969-42a3-8cde-002900e3f9a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:14.560422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48450-0969-42a3-8cde-002900e3f9a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:14.560718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48450-0969-42a3-8cde-002900e3f9a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:14.561057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48450-0969-42a3-8cde-002900e3f9a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:14.563168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d3c48450-0969-42a3-8cde-002900e3f9a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:14.575979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c48450-0969-42a3-8cde-002900e3f9a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:14.578335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c48450-0969-42a3-8cde-002900e3f9a7 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:14.594643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1216/2416] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:14.694302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d01ae3-ffec-4feb-b1b7-e29a0f856655 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:14.695845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-81d01ae3-ffec-4feb-b1b7-e29a0f856655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:14.711141 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d01ae3-ffec-4feb-b1b7-e29a0f856655 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:14.716003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:14.716935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:14.718083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:14.718083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] PATH_INFO: `/v3/projects/db17158c248d477fa4e884bc774c5799` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:14.721868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db17158c248d477fa4e884bc774c5799)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:14.721868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:14.774120 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db17158c248d477fa4e884bc774c5799', 'request_id': 'req-d7698cc5-8690-4add-9bca-e9ed08d3af89'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.780101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db17158c248d477fa4e884bc774c5799', 'request_id': 'req-d7698cc5-8690-4add-9bca-e9ed08d3af89'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.786370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db17158c248d477fa4e884bc774c5799', 'request_id': 'req-d7698cc5-8690-4add-9bca-e9ed08d3af89'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.798675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] The token cache is being invalidate because project db17158c248d477fa4e884bc774c5799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:10:14.799895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.801611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.803750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1201/2417] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:10:14 2022] DELETE /identity/v3/projects/db17158c248d477fa4e884bc774c5799 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:14.807790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48450-0969-42a3-8cde-002900e3f9a7 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:14.821509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48450-0969-42a3-8cde-002900e3f9a7 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:14.880803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:14.881681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:14.881966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:14.882242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] PATH_INFO: `/v3/users/0910f39a1d3e42bbba53e42f8406aa3d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:14.883462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0910f39a1d3e42bbba53e42f8406aa3d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:14.885373 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:14.931598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0910f39a1d3e42bbba53e42f8406aa3d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.944911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0910f39a1d3e42bbba53e42f8406aa3d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.952533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0910f39a1d3e42bbba53e42f8406aa3d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.953366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0910f39a1d3e42bbba53e42f8406aa3d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.959111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:14.961365 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-553f074d-e500-449a-8e42-37721cd5bb35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0910f39a1d3e42bbba53e42f8406aa3d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:14.969725 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1217/2418] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:10:14 2022] DELETE /identity/v3/users/0910f39a1d3e42bbba53e42f8406aa3d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:10:15.262708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:15.265155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:15.280666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7698cc5-8690-4add-9bca-e9ed08d3af89 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:15.322575 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:15.351121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:15.352217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:15.352655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:15.353078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] PATH_INFO: `/v3/projects/1e8a62ef72384bb4a8c351ec103ee3ce` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:15.354994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8a62ef72384bb4a8c351ec103ee3ce)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:15.357998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:15.428967 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8a62ef72384bb4a8c351ec103ee3ce', 'request_id': 'req-189b95b0-02eb-4afd-9759-a71fcca6df04'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:15.435715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8a62ef72384bb4a8c351ec103ee3ce', 'request_id': 'req-189b95b0-02eb-4afd-9759-a71fcca6df04'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:15.442439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8a62ef72384bb4a8c351ec103ee3ce', 'request_id': 'req-189b95b0-02eb-4afd-9759-a71fcca6df04'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:15.460881 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] The token cache is being invalidate because project 1e8a62ef72384bb4a8c351ec103ee3ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:10:15.461335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:15.462389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-189b95b0-02eb-4afd-9759-a71fcca6df04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:15.465541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1202/2419] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:10:15 2022] DELETE /identity/v3/projects/1e8a62ef72384bb4a8c351ec103ee3ce => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:15.470721 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80161f-f0e7-4a63-b301-81486c19fbd5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:15.471237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80161f-f0e7-4a63-b301-81486c19fbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:15.471512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80161f-f0e7-4a63-b301-81486c19fbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:15.492538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ed80161f-f0e7-4a63-b301-81486c19fbd5 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:15.493763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ed80161f-f0e7-4a63-b301-81486c19fbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:15.575528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1218/2420] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:15.584220 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be73e0-b47c-45c3-940a-50da0c1a8669 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:15.584518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be73e0-b47c-45c3-940a-50da0c1a8669 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:15.584803 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be73e0-b47c-45c3-940a-50da0c1a8669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:15.606259 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-52be73e0-b47c-45c3-940a-50da0c1a8669 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:15.607710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-52be73e0-b47c-45c3-940a-50da0c1a8669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:15.619414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1203/2421] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:10:15.624565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80161f-f0e7-4a63-b301-81486c19fbd5 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:15.640043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed80161f-f0e7-4a63-b301-81486c19fbd5 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:15.647910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fda2b3-7d72-479f-9762-6cf5481fb95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:15.648827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fda2b3-7d72-479f-9762-6cf5481fb95a admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:15.649227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fda2b3-7d72-479f-9762-6cf5481fb95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:15.649645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fda2b3-7d72-479f-9762-6cf5481fb95a admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:15.650657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fda2b3-7d72-479f-9762-6cf5481fb95a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:10:15.651079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fda2b3-7d72-479f-9762-6cf5481fb95a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:15.654075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fda2b3-7d72-479f-9762-6cf5481fb95a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:15.669233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1219/2422] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:10:15.674374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be73e0-b47c-45c3-940a-50da0c1a8669 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:15.687423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be73e0-b47c-45c3-940a-50da0c1a8669 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:15.692569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01362f-a6f7-4423-972b-929c0c88f328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:15.694810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01362f-a6f7-4423-972b-929c0c88f328 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:15.695201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01362f-a6f7-4423-972b-929c0c88f328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:15.695549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01362f-a6f7-4423-972b-929c0c88f328 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:15.696324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01362f-a6f7-4423-972b-929c0c88f328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:15.698357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01362f-a6f7-4423-972b-929c0c88f328 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:15.734010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1204/2423] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:10:15 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:15.739809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fda2b3-7d72-479f-9762-6cf5481fb95a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:15.754200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fda2b3-7d72-479f-9762-6cf5481fb95a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:15.804813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6d574-621c-4f29-a466-e2a30e2ae2d2 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:15.805099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6d574-621c-4f29-a466-e2a30e2ae2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:15.805388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6d574-621c-4f29-a466-e2a30e2ae2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:15.830807 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26df269-ae8f-4730-b9cf-8f01d79cdfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:15.831403 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26df269-ae8f-4730-b9cf-8f01d79cdfe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:15.831764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26df269-ae8f-4730-b9cf-8f01d79cdfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:15.832539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26df269-ae8f-4730-b9cf-8f01d79cdfe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:15.832894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-5ea6d574-621c-4f29-a466-e2a30e2ae2d2 None None] MFA Rules not processed for user `4e6db41ec3b94c27bb4af92a35ee0e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:15.833227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26df269-ae8f-4730-b9cf-8f01d79cdfe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:15.834277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5ea6d574-621c-4f29-a466-e2a30e2ae2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:15.835187 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26df269-ae8f-4730-b9cf-8f01d79cdfe2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:15.861777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1220/2424] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:10:15 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:15.863699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26df269-ae8f-4730-b9cf-8f01d79cdfe2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:15.875511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26df269-ae8f-4730-b9cf-8f01d79cdfe2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:15.882022 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3203a6-bc11-4391-adca-866c73311ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:15.882862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3203a6-bc11-4391-adca-866c73311ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:15.883253 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3203a6-bc11-4391-adca-866c73311ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:15.883717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3203a6-bc11-4391-adca-866c73311ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:15.884662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3203a6-bc11-4391-adca-866c73311ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:15.886537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3203a6-bc11-4391-adca-866c73311ae9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:15.900286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1205/2425] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:10:15.905201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1221/2426] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:15.909228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6d574-621c-4f29-a466-e2a30e2ae2d2 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:15.924508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6d574-621c-4f29-a466-e2a30e2ae2d2 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:15.931796 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600850d2-a924-46f4-be2b-7dfe41b1ea35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:15.932584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600850d2-a924-46f4-be2b-7dfe41b1ea35 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:15.933004 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600850d2-a924-46f4-be2b-7dfe41b1ea35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:15.933377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600850d2-a924-46f4-be2b-7dfe41b1ea35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:15.934205 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600850d2-a924-46f4-be2b-7dfe41b1ea35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:15.936053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600850d2-a924-46f4-be2b-7dfe41b1ea35 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:15.950604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1206/2427] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:15.951468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3203a6-bc11-4391-adca-866c73311ae9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:15.963349 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3203a6-bc11-4391-adca-866c73311ae9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:15.969270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed0522-0368-4220-bf00-eb7fdb955b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:15.970590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed0522-0368-4220-bf00-eb7fdb955b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:15.971543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed0522-0368-4220-bf00-eb7fdb955b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:15.971543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed0522-0368-4220-bf00-eb7fdb955b52 admin admin] PATH_INFO: `/v3/projects/2c39d53184ab4bb4b1fee4759188c4a9/users/5242cecfc3c9400caa571f1064964366/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:15.991109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed0522-0368-4220-bf00-eb7fdb955b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c39d53184ab4bb4b1fee4759188c4a9, user_id=5242cecfc3c9400caa571f1064964366, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:15.995112 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed0522-0368-4220-bf00-eb7fdb955b52 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:16.031290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1222/2428] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:10:15 2022] PUT /identity/v3/projects/2c39d53184ab4bb4b1fee4759188c4a9/users/5242cecfc3c9400caa571f1064964366/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:16.043776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78602582-e694-4e39-9b88-c616bbb3a121 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:16.044070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78602582-e694-4e39-9b88-c616bbb3a121 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:16.044349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78602582-e694-4e39-9b88-c616bbb3a121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:16.090354 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-78602582-e694-4e39-9b88-c616bbb3a121 None None] MFA Rules not processed for user `5242cecfc3c9400caa571f1064964366`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:16.092159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78602582-e694-4e39-9b88-c616bbb3a121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:16.127010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:16.217503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1207/2429] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:16.233455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed0522-0368-4220-bf00-eb7fdb955b52 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:16.235683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b7ed0522-0368-4220-bf00-eb7fdb955b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:16.311228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:16.346062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed0522-0368-4220-bf00-eb7fdb955b52 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:16.355211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619a62b-4b23-4ec0-8382-13d4c977039b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3bd8dd462846bf9423f8528aa5c489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:16.356653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619a62b-4b23-4ec0-8382-13d4c977039b service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:16.358852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619a62b-4b23-4ec0-8382-13d4c977039b service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:16.359248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619a62b-4b23-4ec0-8382-13d4c977039b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:16.379987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619a62b-4b23-4ec0-8382-13d4c977039b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:16.385288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619a62b-4b23-4ec0-8382-13d4c977039b service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:16.416241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1223/2430] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:16.608155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78602582-e694-4e39-9b88-c616bbb3a121 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:16.610316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-78602582-e694-4e39-9b88-c616bbb3a121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:16.679156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78602582-e694-4e39-9b88-c616bbb3a121 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:16.684124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3023b5-e2dd-46f8-87a2-5644b2b99f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:16.684811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3023b5-e2dd-46f8-87a2-5644b2b99f0f service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:16.685101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3023b5-e2dd-46f8-87a2-5644b2b99f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:16.685382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3023b5-e2dd-46f8-87a2-5644b2b99f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:16.687186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2c3023b5-e2dd-46f8-87a2-5644b2b99f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:16.698346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3023b5-e2dd-46f8-87a2-5644b2b99f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:16.699892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3023b5-e2dd-46f8-87a2-5644b2b99f0f service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:16.711326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1208/2431] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:17.101821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619a62b-4b23-4ec0-8382-13d4c977039b service glance] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:17.103433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c619a62b-4b23-4ec0-8382-13d4c977039b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:17.209957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619a62b-4b23-4ec0-8382-13d4c977039b service glance] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:17.215666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca300b-89f4-4a54-8776-6178f8eb9bee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:17.216320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca300b-89f4-4a54-8776-6178f8eb9bee service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:17.216685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca300b-89f4-4a54-8776-6178f8eb9bee service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:17.216967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca300b-89f4-4a54-8776-6178f8eb9bee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:17.219245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d8ca300b-89f4-4a54-8776-6178f8eb9bee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:17.232240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca300b-89f4-4a54-8776-6178f8eb9bee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:17.233963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca300b-89f4-4a54-8776-6178f8eb9bee service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:17.245266 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1224/2432] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:19.803122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2afe8-efb8-4d86-b595-1efe27c8646a None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:19.803711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2afe8-efb8-4d86-b595-1efe27c8646a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:19.803808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e2afe8-efb8-4d86-b595-1efe27c8646a None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:19.805554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1209/2433] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:10:19.816144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904c125-902f-4493-9f96-115df0761ff6 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:19.817498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904c125-902f-4493-9f96-115df0761ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:19.817815 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904c125-902f-4493-9f96-115df0761ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:19.839597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5904c125-902f-4493-9f96-115df0761ff6 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:19.840836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5904c125-902f-4493-9f96-115df0761ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:19.913389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1225/2434] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:10:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:19.928673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2afe8-efb8-4d86-b595-1efe27c8646a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:19.930431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e8e2afe8-efb8-4d86-b595-1efe27c8646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:19.952793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e2afe8-efb8-4d86-b595-1efe27c8646a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:19.970115 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8897eb-d030-4e63-96af-5d675fe0d499 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3bd8dd462846bf9423f8528aa5c489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:19.970924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8897eb-d030-4e63-96af-5d675fe0d499 service glance] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:19.971366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8897eb-d030-4e63-96af-5d675fe0d499 service glance] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:19.971832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8897eb-d030-4e63-96af-5d675fe0d499 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:19.985577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8897eb-d030-4e63-96af-5d675fe0d499 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:19.987370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8897eb-d030-4e63-96af-5d675fe0d499 service glance] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:19.999771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1210/2435] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:20.077191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c7b77-b777-440d-a277-5202458b88c1 None None] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:20.077486 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c7b77-b777-440d-a277-5202458b88c1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:20.079468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c7b77-b777-440d-a277-5202458b88c1 None None] PATH_INFO: `/` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:20.081153 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1226/2436] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:10:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:10:20.088870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ce5a4-d02f-4c78-9494-0e516b1c2ece None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:20.089118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ce5a4-d02f-4c78-9494-0e516b1c2ece None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:20.090073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ce5a4-d02f-4c78-9494-0e516b1c2ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:20.119235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-756ce5a4-d02f-4c78-9494-0e516b1c2ece None None] MFA Rules not processed for user `94b8d8b089264b5192a651b41bf2a205`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:20.120466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-756ce5a4-d02f-4c78-9494-0e516b1c2ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:20.129051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1211/2437] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:20.145765 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c7b77-b777-440d-a277-5202458b88c1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:20.158662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c7b77-b777-440d-a277-5202458b88c1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:20.165054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a283a74-ce7c-487e-b16f-aee0d89e9cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:20.165054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a283a74-ce7c-487e-b16f-aee0d89e9cff service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:20.165054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a283a74-ce7c-487e-b16f-aee0d89e9cff service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:20.165582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a283a74-ce7c-487e-b16f-aee0d89e9cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:20.173438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a283a74-ce7c-487e-b16f-aee0d89e9cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:20.175315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a283a74-ce7c-487e-b16f-aee0d89e9cff service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:20.198550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1227/2438] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:20.318317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c67d6-db01-4b2e-b2a7-9b3911c8261f None None] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:20.318659 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c67d6-db01-4b2e-b2a7-9b3911c8261f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:20.319108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c67d6-db01-4b2e-b2a7-9b3911c8261f None None] PATH_INFO: `/` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:20.321536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1212/2439] 10.222.0.57 () {54 vars in 921 bytes} [Thu May 26 21:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:10:20.333613 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4025ba-0916-4762-95db-9080d8409af1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:20.334188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4025ba-0916-4762-95db-9080d8409af1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:20.334664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4025ba-0916-4762-95db-9080d8409af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:20.363054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4c4025ba-0916-4762-95db-9080d8409af1 None None] MFA Rules not processed for user `490d773bf21446789b77c5793a518a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:20.365989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4c4025ba-0916-4762-95db-9080d8409af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:20.391655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1228/2440] 10.222.0.57 () {58 vars in 1018 bytes} [Thu May 26 21:10:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:10:20.408452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c67d6-db01-4b2e-b2a7-9b3911c8261f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:20.425159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c67d6-db01-4b2e-b2a7-9b3911c8261f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:20.430067 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bb1c0-9827-40c3-9285-92365a68aab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:20.430921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bb1c0-9827-40c3-9285-92365a68aab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:20.431315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bb1c0-9827-40c3-9285-92365a68aab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:20.431899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bb1c0-9827-40c3-9285-92365a68aab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:20.445633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bb1c0-9827-40c3-9285-92365a68aab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:20.447231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bb1c0-9827-40c3-9285-92365a68aab7 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:20.460868 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1213/2441] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:25.195596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eebb99-0084-4d6f-a1b1-74be3e801f1f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:25.196129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eebb99-0084-4d6f-a1b1-74be3e801f1f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:25.196221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eebb99-0084-4d6f-a1b1-74be3e801f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:25.219870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-08eebb99-0084-4d6f-a1b1-74be3e801f1f None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:25.221283 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-08eebb99-0084-4d6f-a1b1-74be3e801f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:25.267442 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:25.305968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1229/2442] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:25.312560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bb1c0-9827-40c3-9285-92365a68aab7 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:25.314178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-510bb1c0-9827-40c3-9285-92365a68aab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:25.402162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bb1c0-9827-40c3-9285-92365a68aab7 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:25.413506 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c19865-d7f4-44ee-8a16-542467ff1141 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:25.414197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c19865-d7f4-44ee-8a16-542467ff1141 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:25.414531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c19865-d7f4-44ee-8a16-542467ff1141 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:25.414797 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c19865-d7f4-44ee-8a16-542467ff1141 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:25.423893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c19865-d7f4-44ee-8a16-542467ff1141 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:25.426058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c19865-d7f4-44ee-8a16-542467ff1141 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:25.434728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1214/2443] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:10:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:10:33.501059 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eebb99-0084-4d6f-a1b1-74be3e801f1f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:33.644887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eebb99-0084-4d6f-a1b1-74be3e801f1f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:33.662561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:33.663664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:33.663953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:33.664233 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] PATH_INFO: `/v3/users/5242cecfc3c9400caa571f1064964366` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:33.666175 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5242cecfc3c9400caa571f1064964366)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:33.668227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:33.743623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5242cecfc3c9400caa571f1064964366'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:33.770501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5242cecfc3c9400caa571f1064964366'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:33.792965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5242cecfc3c9400caa571f1064964366'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:33.793959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5242cecfc3c9400caa571f1064964366'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:33.804115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5242cecfc3c9400caa571f1064964366'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:33.816510 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1230/2444] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:10:33 2022] DELETE /identity/v3/users/5242cecfc3c9400caa571f1064964366 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:33.817377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb6519-9480-4514-8576-dbb66ddf07ae None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:33.817867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb6519-9480-4514-8576-dbb66ddf07ae None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:33.817957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb6519-9480-4514-8576-dbb66ddf07ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:33.849980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-34cb6519-9480-4514-8576-dbb66ddf07ae None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:33.851388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-34cb6519-9480-4514-8576-dbb66ddf07ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:33.874918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:33.945191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1215/2445] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:33.955701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:33.961609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:34.064375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32beb89-7fc6-4898-9163-6f370fa50f7a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:34.085928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a019127-0c73-4a2d-a2c9-53fde61af616 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:34.087097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a019127-0c73-4a2d-a2c9-53fde61af616 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.087590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a019127-0c73-4a2d-a2c9-53fde61af616 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.088057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a019127-0c73-4a2d-a2c9-53fde61af616 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.098970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a019127-0c73-4a2d-a2c9-53fde61af616 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:34.100752 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a019127-0c73-4a2d-a2c9-53fde61af616 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:34.111584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1231/2446] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:34.388377 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb6519-9480-4514-8576-dbb66ddf07ae None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:34.390101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-34cb6519-9480-4514-8576-dbb66ddf07ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:34.403874 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb6519-9480-4514-8576-dbb66ddf07ae None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:34.409865 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:34.411262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.411494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.411783 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] PATH_INFO: `/v3/projects/2c39d53184ab4bb4b1fee4759188c4a9` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.412986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c39d53184ab4bb4b1fee4759188c4a9)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:34.414947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:34.462570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c39d53184ab4bb4b1fee4759188c4a9', 'request_id': 'req-53d43387-51c6-49b2-a360-9830812cd0d8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:34.468069 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c39d53184ab4bb4b1fee4759188c4a9', 'request_id': 'req-53d43387-51c6-49b2-a360-9830812cd0d8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:34.474769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c39d53184ab4bb4b1fee4759188c4a9', 'request_id': 'req-53d43387-51c6-49b2-a360-9830812cd0d8'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:34.487453 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] The token cache is being invalidate because project 2c39d53184ab4bb4b1fee4759188c4a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:10:34.487738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:34.488350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-53d43387-51c6-49b2-a360-9830812cd0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:34.489918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1216/2447] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:10:34 2022] DELETE /identity/v3/projects/2c39d53184ab4bb4b1fee4759188c4a9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:10:34.495133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85795bb4-92e3-40c7-ab1b-441e0670f641 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.495431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85795bb4-92e3-40c7-ab1b-441e0670f641 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.495714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85795bb4-92e3-40c7-ab1b-441e0670f641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.513583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-85795bb4-92e3-40c7-ab1b-441e0670f641 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:34.514811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-85795bb4-92e3-40c7-ab1b-441e0670f641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:34.519591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:34.576041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1232/2448] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:34.584554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edad97-ed84-4587-9a32-a52eb99ec9aa None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.584848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edad97-ed84-4587-9a32-a52eb99ec9aa None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.585132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edad97-ed84-4587-9a32-a52eb99ec9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.604070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-93edad97-ed84-4587-9a32-a52eb99ec9aa None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:34.605258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-93edad97-ed84-4587-9a32-a52eb99ec9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:34.613052 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1217/2449] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:34.617282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85795bb4-92e3-40c7-ab1b-441e0670f641 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:34.626999 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85795bb4-92e3-40c7-ab1b-441e0670f641 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:34.632122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad9028-58ea-433e-813b-9bdde37b694d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:34.632799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9028-58ea-433e-813b-9bdde37b694d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.633086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9028-58ea-433e-813b-9bdde37b694d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.633370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9028-58ea-433e-813b-9bdde37b694d admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.634091 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad9028-58ea-433e-813b-9bdde37b694d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:10:34.634411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad9028-58ea-433e-813b-9bdde37b694d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:34.636034 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad9028-58ea-433e-813b-9bdde37b694d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:34.648031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1233/2450] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:10:34.652028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edad97-ed84-4587-9a32-a52eb99ec9aa None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:34.662524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93edad97-ed84-4587-9a32-a52eb99ec9aa None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:34.667257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60afb19-33ab-40bc-a892-8110a40e56d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:34.667905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60afb19-33ab-40bc-a892-8110a40e56d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.668192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60afb19-33ab-40bc-a892-8110a40e56d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.668474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60afb19-33ab-40bc-a892-8110a40e56d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.669154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60afb19-33ab-40bc-a892-8110a40e56d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:34.671102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60afb19-33ab-40bc-a892-8110a40e56d1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:34.698629 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1218/2451] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:10:34 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:34.703145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad9028-58ea-433e-813b-9bdde37b694d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:34.714323 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad9028-58ea-433e-813b-9bdde37b694d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:34.789112 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eec632-ff82-4f2b-985c-82ebf483caa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:34.789790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eec632-ff82-4f2b-985c-82ebf483caa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.790078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eec632-ff82-4f2b-985c-82ebf483caa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.790464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eec632-ff82-4f2b-985c-82ebf483caa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.792487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eec632-ff82-4f2b-985c-82ebf483caa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:34.793111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eec632-ff82-4f2b-985c-82ebf483caa3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:34.816749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1234/2452] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:10:34 2022] POST /identity/v3/users => generated 292 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:34.820954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60afb19-33ab-40bc-a892-8110a40e56d1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:34.832091 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60afb19-33ab-40bc-a892-8110a40e56d1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:34.836751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f438e-5806-4d93-b2ae-6432bbe61cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:34.837482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f438e-5806-4d93-b2ae-6432bbe61cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.837862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f438e-5806-4d93-b2ae-6432bbe61cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.838206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526f438e-5806-4d93-b2ae-6432bbe61cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.839042 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f438e-5806-4d93-b2ae-6432bbe61cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:34.840722 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526f438e-5806-4d93-b2ae-6432bbe61cc1 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:34.851997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1219/2453] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:34.855795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eec632-ff82-4f2b-985c-82ebf483caa3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:34.866680 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eec632-ff82-4f2b-985c-82ebf483caa3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:34.871437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd86c8d-8ecf-4159-825e-918a0260308f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:34.872183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86c8d-8ecf-4159-825e-918a0260308f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.872537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86c8d-8ecf-4159-825e-918a0260308f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.872825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86c8d-8ecf-4159-825e-918a0260308f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.873550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd86c8d-8ecf-4159-825e-918a0260308f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:34.875201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd86c8d-8ecf-4159-825e-918a0260308f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:34.886110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1235/2454] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:34.890321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f438e-5806-4d93-b2ae-6432bbe61cc1 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:34.900277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526f438e-5806-4d93-b2ae-6432bbe61cc1 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:34.904999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ce49b-e68a-404d-9cf6-8b34268bcbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:34.905877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce49b-e68a-404d-9cf6-8b34268bcbfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.906159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce49b-e68a-404d-9cf6-8b34268bcbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.906436 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ce49b-e68a-404d-9cf6-8b34268bcbfd admin admin] PATH_INFO: `/v3/projects/24cf32bd40be4be9a83d0771e4f939fa/users/4ad8d2d3e04b4e06bff2c64bff5a1ac3/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.921429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ce49b-e68a-404d-9cf6-8b34268bcbfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=24cf32bd40be4be9a83d0771e4f939fa, user_id=4ad8d2d3e04b4e06bff2c64bff5a1ac3, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:34.924244 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ce49b-e68a-404d-9cf6-8b34268bcbfd admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:34.938721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1220/2455] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:10:34 2022] PUT /identity/v3/projects/24cf32bd40be4be9a83d0771e4f939fa/users/4ad8d2d3e04b4e06bff2c64bff5a1ac3/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:34.944199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feabb4-c19a-478b-a7d6-caeeb8f044d5 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:34.944502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feabb4-c19a-478b-a7d6-caeeb8f044d5 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:34.944787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9feabb4-c19a-478b-a7d6-caeeb8f044d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:34.980380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c9feabb4-c19a-478b-a7d6-caeeb8f044d5 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:34.982155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c9feabb4-c19a-478b-a7d6-caeeb8f044d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:35.042004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1236/2456] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:35.051864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ce49b-e68a-404d-9cf6-8b34268bcbfd admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:35.053587 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-858ce49b-e68a-404d-9cf6-8b34268bcbfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:35.098177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:35.111366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ce49b-e68a-404d-9cf6-8b34268bcbfd admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:35.115501 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c1f58-44a7-4d52-a8df-0b7e60701555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:35.116397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c1f58-44a7-4d52-a8df-0b7e60701555 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:35.116678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c1f58-44a7-4d52-a8df-0b7e60701555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:35.116951 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c1f58-44a7-4d52-a8df-0b7e60701555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:35.125920 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c1f58-44a7-4d52-a8df-0b7e60701555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:35.127389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9c1f58-44a7-4d52-a8df-0b7e60701555 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:35.137890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1221/2457] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:35.836860 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1687cf-15ed-4ed1-86c3-f8f62bddd63c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:35.837356 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1687cf-15ed-4ed1-86c3-f8f62bddd63c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:35.837475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1687cf-15ed-4ed1-86c3-f8f62bddd63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:35.865536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1b1687cf-15ed-4ed1-86c3-f8f62bddd63c None None] MFA Rules not processed for user `4ad8d2d3e04b4e06bff2c64bff5a1ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:35.867078 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1b1687cf-15ed-4ed1-86c3-f8f62bddd63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:35.872506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:35.935691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1237/2458] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:35.946852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c1f58-44a7-4d52-a8df-0b7e60701555 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:35.948349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-bc9c1f58-44a7-4d52-a8df-0b7e60701555 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:36.028180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c1f58-44a7-4d52-a8df-0b7e60701555 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:36.036173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95546d86-a00c-4134-b527-2a88de68f2c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:36.037078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95546d86-a00c-4134-b527-2a88de68f2c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:36.037368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95546d86-a00c-4134-b527-2a88de68f2c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:36.037650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95546d86-a00c-4134-b527-2a88de68f2c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:36.047063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95546d86-a00c-4134-b527-2a88de68f2c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:36.048516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95546d86-a00c-4134-b527-2a88de68f2c5 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:36.058814 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1222/2459] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:37.445684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6b9f-052e-49a0-939f-42c6a1e4d9c1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:37.446202 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6b9f-052e-49a0-939f-42c6a1e4d9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:37.446300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6b9f-052e-49a0-939f-42c6a1e4d9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:37.478674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-895f6b9f-052e-49a0-939f-42c6a1e4d9c1 None None] MFA Rules not processed for user `4ad8d2d3e04b4e06bff2c64bff5a1ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:37.478674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-895f6b9f-052e-49a0-939f-42c6a1e4d9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:37.481581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1238/2460] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:37.532916 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36725081-c2c7-4535-8299-bf0405f0b78c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:37.533382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36725081-c2c7-4535-8299-bf0405f0b78c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:37.533546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36725081-c2c7-4535-8299-bf0405f0b78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:37.555125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-36725081-c2c7-4535-8299-bf0405f0b78c None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:37.556499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-36725081-c2c7-4535-8299-bf0405f0b78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:37.623463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1223/2461] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 26 21:10:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:37.635824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f6b9f-052e-49a0-939f-42c6a1e4d9c1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:37.637207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-895f6b9f-052e-49a0-939f-42c6a1e4d9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:37.741837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f6b9f-052e-49a0-939f-42c6a1e4d9c1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:37.750320 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22401c9a-6ca4-42ac-ad5b-f4541a74332b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:37.751545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22401c9a-6ca4-42ac-ad5b-f4541a74332b service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:37.752104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22401c9a-6ca4-42ac-ad5b-f4541a74332b service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:37.752777 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22401c9a-6ca4-42ac-ad5b-f4541a74332b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:37.766038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22401c9a-6ca4-42ac-ad5b-f4541a74332b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:37.769804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22401c9a-6ca4-42ac-ad5b-f4541a74332b service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:37.782679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1239/2462] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:10:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 21:10:38.017213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36725081-c2c7-4535-8299-bf0405f0b78c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:38.026980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36725081-c2c7-4535-8299-bf0405f0b78c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:38.032518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383a206-5596-47a2-91e0-a2b528aa08f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:38.033353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383a206-5596-47a2-91e0-a2b528aa08f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:38.033839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383a206-5596-47a2-91e0-a2b528aa08f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:38.034307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383a206-5596-47a2-91e0-a2b528aa08f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:38.036325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b383a206-5596-47a2-91e0-a2b528aa08f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:38.047660 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383a206-5596-47a2-91e0-a2b528aa08f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:38.049309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383a206-5596-47a2-91e0-a2b528aa08f5 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:38.059045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1224/2463] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:39.059962 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22401c9a-6ca4-42ac-ad5b-f4541a74332b service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:39.078774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22401c9a-6ca4-42ac-ad5b-f4541a74332b service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:39.083711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc6f54-edd8-4d9f-840c-58ec7c33d394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:39.084651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc6f54-edd8-4d9f-840c-58ec7c33d394 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:39.085106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc6f54-edd8-4d9f-840c-58ec7c33d394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:39.085572 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc6f54-edd8-4d9f-840c-58ec7c33d394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:39.097537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-acbc6f54-edd8-4d9f-840c-58ec7c33d394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:39.228132 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc6f54-edd8-4d9f-840c-58ec7c33d394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:39.229603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc6f54-edd8-4d9f-840c-58ec7c33d394 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:39.262652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1240/2464] 10.222.0.57 () {60 vars in 1396 bytes} [Thu May 26 21:10:39 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:42.438319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51887d1c-7327-43b3-983d-a273c084c99c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:42.438882 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51887d1c-7327-43b3-983d-a273c084c99c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:42.438998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51887d1c-7327-43b3-983d-a273c084c99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:42.463862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-51887d1c-7327-43b3-983d-a273c084c99c None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:42.465213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-51887d1c-7327-43b3-983d-a273c084c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:42.479871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1225/2465] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 26 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:42.496497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc6f54-edd8-4d9f-840c-58ec7c33d394 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:42.516265 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc6f54-edd8-4d9f-840c-58ec7c33d394 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:42.523030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c2b12-4649-4741-bd4e-92453bd24161 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:42.523794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c2b12-4649-4741-bd4e-92453bd24161 service swift] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:42.524145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c2b12-4649-4741-bd4e-92453bd24161 service swift] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:42.524491 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c2b12-4649-4741-bd4e-92453bd24161 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:42.535636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c2b12-4649-4741-bd4e-92453bd24161 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:42.538174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c2b12-4649-4741-bd4e-92453bd24161 service swift] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:42.549837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1241/2466] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:10:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:10:43.484200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51887d1c-7327-43b3-983d-a273c084c99c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:43.485822 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-51887d1c-7327-43b3-983d-a273c084c99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:43.585685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51887d1c-7327-43b3-983d-a273c084c99c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:43.594492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f490995-378e-4ee8-9da9-b197e1dec6f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:43.595213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f490995-378e-4ee8-9da9-b197e1dec6f5 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:43.595510 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f490995-378e-4ee8-9da9-b197e1dec6f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:43.595796 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f490995-378e-4ee8-9da9-b197e1dec6f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:43.603937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7f490995-378e-4ee8-9da9-b197e1dec6f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:43.621663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f490995-378e-4ee8-9da9-b197e1dec6f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:43.623661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f490995-378e-4ee8-9da9-b197e1dec6f5 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:43.636746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1226/2467] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:10:47.349064 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c2b12-4649-4741-bd4e-92453bd24161 service swift] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:47.350727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fa4c2b12-4649-4741-bd4e-92453bd24161 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:47.365087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c2b12-4649-4741-bd4e-92453bd24161 service swift] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:47.370234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:47.371155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:47.371441 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:47.371717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] PATH_INFO: `/v3/users/6917bcbeaa204b159efc6cf456465c2a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:47.372891 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6917bcbeaa204b159efc6cf456465c2a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:47.374845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:47.380254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:47.421590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6917bcbeaa204b159efc6cf456465c2a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.434271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6917bcbeaa204b159efc6cf456465c2a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.441612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6917bcbeaa204b159efc6cf456465c2a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.442239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6917bcbeaa204b159efc6cf456465c2a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.449119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6917bcbeaa204b159efc6cf456465c2a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.455129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1242/2468] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:10:47 2022] DELETE /identity/v3/users/6917bcbeaa204b159efc6cf456465c2a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:47.638037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f490995-378e-4ee8-9da9-b197e1dec6f5 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:47.645253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=100 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:47.652912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f490995-378e-4ee8-9da9-b197e1dec6f5 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:47.713432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:47.714295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:47.714615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:47.714894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] PATH_INFO: `/v3/projects/cdb0cbaf75ee402091c3eec9dd512a2d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:47.716081 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdb0cbaf75ee402091c3eec9dd512a2d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:47.718036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:47.773957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:47.775423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdb0cbaf75ee402091c3eec9dd512a2d', 'request_id': 'req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.775690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:47.781096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdb0cbaf75ee402091c3eec9dd512a2d', 'request_id': 'req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.788049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdb0cbaf75ee402091c3eec9dd512a2d', 'request_id': 'req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.804135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] The token cache is being invalidate because project cdb0cbaf75ee402091c3eec9dd512a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:10:47.804135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.804135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.804673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1227/2469] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:10:47 2022] DELETE /identity/v3/projects/cdb0cbaf75ee402091c3eec9dd512a2d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:47.808197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:47.813428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:47.838865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66e5b5-a6d6-4ddf-a3e1-f137b2448373 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:47.872343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78b30c-fbc8-4515-816e-dabb4f2ff56a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:47.877239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:47.878146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:47.878435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:47.878739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] PATH_INFO: `/v3/users/43ed9f89c496438994c24baaf990062c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:47.879949 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ed9f89c496438994c24baaf990062c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:47.881904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:47.888816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc773-3755-4d24-bfb0-aad66357e61a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:47.889465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc773-3755-4d24-bfb0-aad66357e61a service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:47.889747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc773-3755-4d24-bfb0-aad66357e61a service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:47.890020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fc773-3755-4d24-bfb0-aad66357e61a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:47.891782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8b8fc773-3755-4d24-bfb0-aad66357e61a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:47.904985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fc773-3755-4d24-bfb0-aad66357e61a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:47.906406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fc773-3755-4d24-bfb0-aad66357e61a service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:47.917726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1243/2470] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:47.921013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:47.932762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ed9f89c496438994c24baaf990062c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.933374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ed9f89c496438994c24baaf990062c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.942165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ed9f89c496438994c24baaf990062c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.942990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ed9f89c496438994c24baaf990062c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.956008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ed9f89c496438994c24baaf990062c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:47.968479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1228/2471] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:10:47 2022] DELETE /identity/v3/users/43ed9f89c496438994c24baaf990062c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:10:48.582713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc773-3755-4d24-bfb0-aad66357e61a service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:48.592688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fc773-3755-4d24-bfb0-aad66357e61a service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:48.644412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:48.681691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:48.682965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:48.683894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:48.684181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] PATH_INFO: `/v3/projects/063d0056b740469fb8152f77df552a6a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:48.687835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=063d0056b740469fb8152f77df552a6a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:48.691454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:48.753579 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063d0056b740469fb8152f77df552a6a', 'request_id': 'req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:48.760277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063d0056b740469fb8152f77df552a6a', 'request_id': 'req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:48.766828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063d0056b740469fb8152f77df552a6a', 'request_id': 'req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:48.785232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] The token cache is being invalidate because project 063d0056b740469fb8152f77df552a6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:10:48.785530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:48.786818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:48.788935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1244/2472] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:10:48 2022] DELETE /identity/v3/projects/063d0056b740469fb8152f77df552a6a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:51.461961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:51.464132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:51.486152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ec6a6-5300-4b00-bc0b-f75e366e62c6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:51.546588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:51.547455 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:51.547915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:51.548192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] PATH_INFO: `/v3/users/83f3cfb67d3147d7b856158dcf32ff03` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:51.549397 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f3cfb67d3147d7b856158dcf32ff03)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:51.551615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:51.618098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f3cfb67d3147d7b856158dcf32ff03'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:51.618840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f3cfb67d3147d7b856158dcf32ff03'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:51.626065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f3cfb67d3147d7b856158dcf32ff03'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:51.626714 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f3cfb67d3147d7b856158dcf32ff03'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:51.640417 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-1617415f-2ea7-434d-aaf6-ac61a40a8fe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f3cfb67d3147d7b856158dcf32ff03'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:51.653838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1229/2473] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:10:51 2022] DELETE /identity/v3/users/83f3cfb67d3147d7b856158dcf32ff03 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:51.852295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:51.853817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:51.868039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc412f0-df5a-40a8-b4a0-70a2a2fbc8d1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:51.927710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:51.928570 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:51.928855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:51.929134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] PATH_INFO: `/v3/projects/aee8ddc72b8a4baeb01503afbd8aeab1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:51.930514 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee8ddc72b8a4baeb01503afbd8aeab1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:51.932467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:51.980058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee8ddc72b8a4baeb01503afbd8aeab1', 'request_id': 'req-9fca59bf-8547-43b9-9411-ed1d4c46d64b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:51.986638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee8ddc72b8a4baeb01503afbd8aeab1', 'request_id': 'req-9fca59bf-8547-43b9-9411-ed1d4c46d64b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:51.992722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee8ddc72b8a4baeb01503afbd8aeab1', 'request_id': 'req-9fca59bf-8547-43b9-9411-ed1d4c46d64b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:52.006736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] The token cache is being invalidate because project aee8ddc72b8a4baeb01503afbd8aeab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:10:52.007030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:52.007670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9fca59bf-8547-43b9-9411-ed1d4c46d64b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:10:52.010507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1245/2474] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:10:51 2022] DELETE /identity/v3/projects/aee8ddc72b8a4baeb01503afbd8aeab1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:52.014144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a8f0b-8c67-40d6-9dd5-eac7c61f0fad None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.014541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a8f0b-8c67-40d6-9dd5-eac7c61f0fad None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.014896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a8f0b-8c67-40d6-9dd5-eac7c61f0fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.035731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f25a8f0b-8c67-40d6-9dd5-eac7c61f0fad None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:52.036991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f25a8f0b-8c67-40d6-9dd5-eac7c61f0fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:52.063933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:52.100328 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1230/2475] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:52.108241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54287f-fdc0-4e33-acd6-c8001db53047 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.108538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54287f-fdc0-4e33-acd6-c8001db53047 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.108825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54287f-fdc0-4e33-acd6-c8001db53047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.119115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:52.132366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1c54287f-fdc0-4e33-acd6-c8001db53047 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:52.133578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1c54287f-fdc0-4e33-acd6-c8001db53047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:52.141810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1246/2476] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:52.146636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a8f0b-8c67-40d6-9dd5-eac7c61f0fad None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.156810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a8f0b-8c67-40d6-9dd5-eac7c61f0fad None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.161720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b0733-00aa-44e8-a018-82fda2da44a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.162402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b0733-00aa-44e8-a018-82fda2da44a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.162718 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b0733-00aa-44e8-a018-82fda2da44a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.163021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b0733-00aa-44e8-a018-82fda2da44a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.163807 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b0733-00aa-44e8-a018-82fda2da44a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:10:52.164353 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b0733-00aa-44e8-a018-82fda2da44a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.166711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b0733-00aa-44e8-a018-82fda2da44a9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.181054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1231/2477] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:10:52.185304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54287f-fdc0-4e33-acd6-c8001db53047 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.195896 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54287f-fdc0-4e33-acd6-c8001db53047 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.200715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c510a57-3041-4434-ac31-8deb65574628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.201384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c510a57-3041-4434-ac31-8deb65574628 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.201669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c510a57-3041-4434-ac31-8deb65574628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.201948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c510a57-3041-4434-ac31-8deb65574628 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.202678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c510a57-3041-4434-ac31-8deb65574628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.204618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c510a57-3041-4434-ac31-8deb65574628 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.242094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1247/2478] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:10:52 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:52.246483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b0733-00aa-44e8-a018-82fda2da44a9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.257364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b0733-00aa-44e8-a018-82fda2da44a9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.315886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304ec08-d370-4d8a-83db-0914eaa3491b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.316554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ec08-d370-4d8a-83db-0914eaa3491b admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.316838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ec08-d370-4d8a-83db-0914eaa3491b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.317831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304ec08-d370-4d8a-83db-0914eaa3491b admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.318552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304ec08-d370-4d8a-83db-0914eaa3491b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.320502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304ec08-d370-4d8a-83db-0914eaa3491b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.340231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1232/2479] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:10:52 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:52.344479 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c510a57-3041-4434-ac31-8deb65574628 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.354686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c510a57-3041-4434-ac31-8deb65574628 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.359321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e426f8-a182-4424-a38c-05fe1ad697c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.359967 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e426f8-a182-4424-a38c-05fe1ad697c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.360248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e426f8-a182-4424-a38c-05fe1ad697c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.360522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e426f8-a182-4424-a38c-05fe1ad697c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.361232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e426f8-a182-4424-a38c-05fe1ad697c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.363350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e426f8-a182-4424-a38c-05fe1ad697c2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.373902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1248/2480] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:52.378489 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304ec08-d370-4d8a-83db-0914eaa3491b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.388855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304ec08-d370-4d8a-83db-0914eaa3491b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.393451 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe317eb-e914-48cc-bc06-3dfb8c2a8bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.394101 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe317eb-e914-48cc-bc06-3dfb8c2a8bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.394383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe317eb-e914-48cc-bc06-3dfb8c2a8bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.394694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe317eb-e914-48cc-bc06-3dfb8c2a8bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.395408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe317eb-e914-48cc-bc06-3dfb8c2a8bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.396999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe317eb-e914-48cc-bc06-3dfb8c2a8bcc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.407627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1233/2481] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:52.411950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e426f8-a182-4424-a38c-05fe1ad697c2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.421767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e426f8-a182-4424-a38c-05fe1ad697c2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.426851 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d3b53-42a6-4ec9-8559-0505adab5e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.427748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d3b53-42a6-4ec9-8559-0505adab5e73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.428028 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d3b53-42a6-4ec9-8559-0505adab5e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.428303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d3b53-42a6-4ec9-8559-0505adab5e73 admin admin] PATH_INFO: `/v3/projects/58dd4906c578482aa90d5612e58be77a/users/bfba52a9217c4b68990908b4d0f9ed95/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.444515 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d3b53-42a6-4ec9-8559-0505adab5e73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58dd4906c578482aa90d5612e58be77a, user_id=bfba52a9217c4b68990908b4d0f9ed95, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.447217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878d3b53-42a6-4ec9-8559-0505adab5e73 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.461005 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1249/2482] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:10:52 2022] PUT /identity/v3/projects/58dd4906c578482aa90d5612e58be77a/users/bfba52a9217c4b68990908b4d0f9ed95/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:52.472746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7edcb-01a8-48ad-82f6-08f24c5e14f6 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.473270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7edcb-01a8-48ad-82f6-08f24c5e14f6 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.473408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7edcb-01a8-48ad-82f6-08f24c5e14f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.501918 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ddc7edcb-01a8-48ad-82f6-08f24c5e14f6 None None] MFA Rules not processed for user `bfba52a9217c4b68990908b4d0f9ed95`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:52.503157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ddc7edcb-01a8-48ad-82f6-08f24c5e14f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:52.566581 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1234/2483] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:52.571145 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d3b53-42a6-4ec9-8559-0505adab5e73 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.639626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d3b53-42a6-4ec9-8559-0505adab5e73 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.644384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05a3ee-374e-49d2-8d90-dcee2a120dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.645059 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05a3ee-374e-49d2-8d90-dcee2a120dae admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.645355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05a3ee-374e-49d2-8d90-dcee2a120dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.645635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05a3ee-374e-49d2-8d90-dcee2a120dae admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.646346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05a3ee-374e-49d2-8d90-dcee2a120dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.648303 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05a3ee-374e-49d2-8d90-dcee2a120dae admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.678926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1250/2484] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:10:52 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:52.682829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7edcb-01a8-48ad-82f6-08f24c5e14f6 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.693466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7edcb-01a8-48ad-82f6-08f24c5e14f6 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.718685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:52.751935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e05ee-66a0-4f07-bd99-ce6040183f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.752596 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e05ee-66a0-4f07-bd99-ce6040183f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.752886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e05ee-66a0-4f07-bd99-ce6040183f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.753166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e05ee-66a0-4f07-bd99-ce6040183f77 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.753861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e05ee-66a0-4f07-bd99-ce6040183f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.755833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e05ee-66a0-4f07-bd99-ce6040183f77 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.780118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1235/2485] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:10:52 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:10:52.781455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05a3ee-374e-49d2-8d90-dcee2a120dae admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.794094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05a3ee-374e-49d2-8d90-dcee2a120dae admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.804831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355de05-eedc-46be-8840-a77e4aa1aa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.804831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355de05-eedc-46be-8840-a77e4aa1aa33 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.804831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355de05-eedc-46be-8840-a77e4aa1aa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.804831 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355de05-eedc-46be-8840-a77e4aa1aa33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.805269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355de05-eedc-46be-8840-a77e4aa1aa33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.805269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d355de05-eedc-46be-8840-a77e4aa1aa33 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.824626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1251/2486] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:10:52.828723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e05ee-66a0-4f07-bd99-ce6040183f77 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.839577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e05ee-66a0-4f07-bd99-ce6040183f77 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.844186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4911c-35cf-4a8b-bc6a-b7da4d0c5065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.845053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4911c-35cf-4a8b-bc6a-b7da4d0c5065 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.845332 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4911c-35cf-4a8b-bc6a-b7da4d0c5065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.845604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4911c-35cf-4a8b-bc6a-b7da4d0c5065 admin admin] PATH_INFO: `/v3/projects/09cfe5d6ced645c8baffb2a1d9b15ecf/users/5fd5748794144bb4843c5b5ecd9b63c1/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.860763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4911c-35cf-4a8b-bc6a-b7da4d0c5065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cfe5d6ced645c8baffb2a1d9b15ecf, user_id=5fd5748794144bb4843c5b5ecd9b63c1, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.863523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4911c-35cf-4a8b-bc6a-b7da4d0c5065 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.877001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1236/2487] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:10:52 2022] PUT /identity/v3/projects/09cfe5d6ced645c8baffb2a1d9b15ecf/users/5fd5748794144bb4843c5b5ecd9b63c1/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:10:52.880952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355de05-eedc-46be-8840-a77e4aa1aa33 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.890844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d355de05-eedc-46be-8840-a77e4aa1aa33 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.946770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09acabc-dd6d-4c84-918f-daf817e2b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.947440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09acabc-dd6d-4c84-918f-daf817e2b071 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.947727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09acabc-dd6d-4c84-918f-daf817e2b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.948005 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09acabc-dd6d-4c84-918f-daf817e2b071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.948722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09acabc-dd6d-4c84-918f-daf817e2b071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.950328 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09acabc-dd6d-4c84-918f-daf817e2b071 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.960722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1252/2488] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:10:52.965059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4911c-35cf-4a8b-bc6a-b7da4d0c5065 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:52.976794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4911c-35cf-4a8b-bc6a-b7da4d0c5065 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:52.981404 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d1d1f-3be7-4603-b650-85f107d4be8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:52.982284 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1d1f-3be7-4603-b650-85f107d4be8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:52.982628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1d1f-3be7-4603-b650-85f107d4be8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:52.982922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d1d1f-3be7-4603-b650-85f107d4be8a admin admin] PATH_INFO: `/v3/projects/09cfe5d6ced645c8baffb2a1d9b15ecf/users/5fd5748794144bb4843c5b5ecd9b63c1/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:52.984856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d1d1f-3be7-4603-b650-85f107d4be8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cfe5d6ced645c8baffb2a1d9b15ecf, user_id=5fd5748794144bb4843c5b5ecd9b63c1, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:52.987635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d1d1f-3be7-4603-b650-85f107d4be8a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:52.998418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1237/2489] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:10:52 2022] PUT /identity/v3/projects/09cfe5d6ced645c8baffb2a1d9b15ecf/users/5fd5748794144bb4843c5b5ecd9b63c1/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:10:53.008589 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa52a82-631f-466b-bc0f-6048d7a9a82c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:53.008883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa52a82-631f-466b-bc0f-6048d7a9a82c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:53.009168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa52a82-631f-466b-bc0f-6048d7a9a82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:53.015031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:10:53.038147 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-caa52a82-631f-466b-bc0f-6048d7a9a82c None None] MFA Rules not processed for user `5fd5748794144bb4843c5b5ecd9b63c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:10:53.039406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-caa52a82-631f-466b-bc0f-6048d7a9a82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:53.107637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1253/2490] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:10:53 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:10:53.119414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d1d1f-3be7-4603-b650-85f107d4be8a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:53.121175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-981d1d1f-3be7-4603-b650-85f107d4be8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:10:53.179622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d1d1f-3be7-4603-b650-85f107d4be8a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:53.184103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05797c0-2ff9-4b0a-8cc6-e7aca8bdbeff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:53.184768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05797c0-2ff9-4b0a-8cc6-e7aca8bdbeff service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:53.185129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05797c0-2ff9-4b0a-8cc6-e7aca8bdbeff service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:53.185648 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05797c0-2ff9-4b0a-8cc6-e7aca8bdbeff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:53.196269 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05797c0-2ff9-4b0a-8cc6-e7aca8bdbeff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:53.197765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05797c0-2ff9-4b0a-8cc6-e7aca8bdbeff service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:53.208529 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1238/2491] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:53 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:10:53.282646 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa52a82-631f-466b-bc0f-6048d7a9a82c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:10:53.292417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa52a82-631f-466b-bc0f-6048d7a9a82c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:10:53.296082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056eeab1-a596-46a8-a35f-8999b4f3466e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:10:53.296717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eeab1-a596-46a8-a35f-8999b4f3466e service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:10:53.297003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eeab1-a596-46a8-a35f-8999b4f3466e service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:10:53.297278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eeab1-a596-46a8-a35f-8999b4f3466e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:10:53.359305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056eeab1-a596-46a8-a35f-8999b4f3466e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:10:53.360740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056eeab1-a596-46a8-a35f-8999b4f3466e service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:10:53.369714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1254/2492] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:10:53 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:03.203434 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=58,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:03.365697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=61 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:03.532356 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05797c0-2ff9-4b0a-8cc6-e7aca8bdbeff service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:03.592239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05797c0-2ff9-4b0a-8cc6-e7aca8bdbeff service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:03.596742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:03.597590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:03.597871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:03.598146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] PATH_INFO: `/v3/users/bfba52a9217c4b68990908b4d0f9ed95` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:03.599364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfba52a9217c4b68990908b4d0f9ed95)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:03.601289 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:03.649648 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfba52a9217c4b68990908b4d0f9ed95'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:03.650323 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfba52a9217c4b68990908b4d0f9ed95'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:03.657329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfba52a9217c4b68990908b4d0f9ed95'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:03.657935 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfba52a9217c4b68990908b4d0f9ed95'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:03.669973 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfba52a9217c4b68990908b4d0f9ed95'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:03.682008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1239/2493] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:03 2022] DELETE /identity/v3/users/bfba52a9217c4b68990908b4d0f9ed95 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:11:03.686644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3faf5c-da70-4d42-90e6-f2d4a4c83879 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:03.686937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3faf5c-da70-4d42-90e6-f2d4a4c83879 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:03.687218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3faf5c-da70-4d42-90e6-f2d4a4c83879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:03.703923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-eb3faf5c-da70-4d42-90e6-f2d4a4c83879 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:03.705615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eb3faf5c-da70-4d42-90e6-f2d4a4c83879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:03.763407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1255/2494] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:03.774649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:03.776038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:03.833880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0fb34-5177-49f0-9e05-0dd3dce57a92 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:03.837516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44c3c8-298b-49e7-9ad1-a32cda95db24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:03.838162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44c3c8-298b-49e7-9ad1-a32cda95db24 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:03.838464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44c3c8-298b-49e7-9ad1-a32cda95db24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:03.838764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44c3c8-298b-49e7-9ad1-a32cda95db24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:03.848414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44c3c8-298b-49e7-9ad1-a32cda95db24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:03.850166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44c3c8-298b-49e7-9ad1-a32cda95db24 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:03.859790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1240/2495] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:11:04.121166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3faf5c-da70-4d42-90e6-f2d4a4c83879 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:04.122621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-eb3faf5c-da70-4d42-90e6-f2d4a4c83879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:04.135271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3faf5c-da70-4d42-90e6-f2d4a4c83879 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:04.140313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:04.141144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:04.141429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:04.141707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] PATH_INFO: `/v3/projects/58dd4906c578482aa90d5612e58be77a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:04.144564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58dd4906c578482aa90d5612e58be77a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:04.146540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:04.198352 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58dd4906c578482aa90d5612e58be77a', 'request_id': 'req-3095bccc-3b44-4b03-b0bd-6396d8f5f284'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.204535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58dd4906c578482aa90d5612e58be77a', 'request_id': 'req-3095bccc-3b44-4b03-b0bd-6396d8f5f284'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.210114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58dd4906c578482aa90d5612e58be77a', 'request_id': 'req-3095bccc-3b44-4b03-b0bd-6396d8f5f284'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.222435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] The token cache is being invalidate because project 58dd4906c578482aa90d5612e58be77a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:04.222817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.223432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.225040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1256/2496] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:04 2022] DELETE /identity/v3/projects/58dd4906c578482aa90d5612e58be77a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:04.228938 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44c3c8-298b-49e7-9ad1-a32cda95db24 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:04.238972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44c3c8-298b-49e7-9ad1-a32cda95db24 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:04.257239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:04.296468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:04.297253 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:04.297538 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:04.297815 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] PATH_INFO: `/v3/users/5fd5748794144bb4843c5b5ecd9b63c1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:04.299539 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd5748794144bb4843c5b5ecd9b63c1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:04.301470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:04.345378 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd5748794144bb4843c5b5ecd9b63c1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.346032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd5748794144bb4843c5b5ecd9b63c1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.354493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd5748794144bb4843c5b5ecd9b63c1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.355104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd5748794144bb4843c5b5ecd9b63c1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.367982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-dd65fee5-cfc9-41fc-ac3f-8c3c535f67cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd5748794144bb4843c5b5ecd9b63c1'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.380543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1241/2497] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:04 2022] DELETE /identity/v3/users/5fd5748794144bb4843c5b5ecd9b63c1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:04.649062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:04.650658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:04.664181 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095bccc-3b44-4b03-b0bd-6396d8f5f284 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:04.723190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:04.724056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:04.724342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:04.724620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] PATH_INFO: `/v3/projects/09cfe5d6ced645c8baffb2a1d9b15ecf` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:04.725794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] RBAC: Authorizing `identity:delete_project(project_id=09cfe5d6ced645c8baffb2a1d9b15ecf)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:04.727778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:04.758711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:04.774793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09cfe5d6ced645c8baffb2a1d9b15ecf', 'request_id': 'req-0b5fc710-1b6d-4190-950d-8510010b555f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.780684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09cfe5d6ced645c8baffb2a1d9b15ecf', 'request_id': 'req-0b5fc710-1b6d-4190-950d-8510010b555f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.786598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09cfe5d6ced645c8baffb2a1d9b15ecf', 'request_id': 'req-0b5fc710-1b6d-4190-950d-8510010b555f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.798311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] The token cache is being invalidate because project 09cfe5d6ced645c8baffb2a1d9b15ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:04.798635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.799991 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0b5fc710-1b6d-4190-950d-8510010b555f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:04.801553 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1257/2498] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:04 2022] DELETE /identity/v3/projects/09cfe5d6ced645c8baffb2a1d9b15ecf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:04.807197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7d565-8cd6-4d39-9d24-a3aa14259356 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:04.807491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7d565-8cd6-4d39-9d24-a3aa14259356 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:04.807773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7d565-8cd6-4d39-9d24-a3aa14259356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:04.825297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-81c7d565-8cd6-4d39-9d24-a3aa14259356 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:04.826490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-81c7d565-8cd6-4d39-9d24-a3aa14259356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:04.884055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1242/2499] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:04.891901 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f29b80-945d-4d60-98b7-0d673b80132d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:04.892213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f29b80-945d-4d60-98b7-0d673b80132d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:04.892496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f29b80-945d-4d60-98b7-0d673b80132d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:04.912356 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f8f29b80-945d-4d60-98b7-0d673b80132d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:04.913541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f8f29b80-945d-4d60-98b7-0d673b80132d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:04.921272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1258/2500] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:04.925308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7d565-8cd6-4d39-9d24-a3aa14259356 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:04.935118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7d565-8cd6-4d39-9d24-a3aa14259356 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:04.939671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6f5e8-3ad5-4097-8b47-e84474a6f474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:04.940316 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6f5e8-3ad5-4097-8b47-e84474a6f474 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:04.940603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6f5e8-3ad5-4097-8b47-e84474a6f474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:04.940883 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6f5e8-3ad5-4097-8b47-e84474a6f474 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:04.941604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6f5e8-3ad5-4097-8b47-e84474a6f474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:04.941913 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6f5e8-3ad5-4097-8b47-e84474a6f474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:04.943566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e6f5e8-3ad5-4097-8b47-e84474a6f474 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:04.955009 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1243/2501] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:04.959402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f29b80-945d-4d60-98b7-0d673b80132d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:04.968356 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f29b80-945d-4d60-98b7-0d673b80132d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:04.973604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabdece-4d7a-4706-9031-bfcc2b795029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:04.974239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabdece-4d7a-4706-9031-bfcc2b795029 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:04.974559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabdece-4d7a-4706-9031-bfcc2b795029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:04.974835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabdece-4d7a-4706-9031-bfcc2b795029 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:04.975517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabdece-4d7a-4706-9031-bfcc2b795029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:04.977379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabdece-4d7a-4706-9031-bfcc2b795029 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.001793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1259/2502] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:04 2022] POST /identity/v3/projects => generated 399 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:05.005856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6f5e8-3ad5-4097-8b47-e84474a6f474 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.015709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6f5e8-3ad5-4097-8b47-e84474a6f474 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.041692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:05.072389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef7fb4-93ba-4b68-8e17-bc522112303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.073048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef7fb4-93ba-4b68-8e17-bc522112303a admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.073334 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef7fb4-93ba-4b68-8e17-bc522112303a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.073614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef7fb4-93ba-4b68-8e17-bc522112303a admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.074309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef7fb4-93ba-4b68-8e17-bc522112303a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.076339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef7fb4-93ba-4b68-8e17-bc522112303a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.096454 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1244/2503] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:05 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:05.100293 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabdece-4d7a-4706-9031-bfcc2b795029 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.110498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabdece-4d7a-4706-9031-bfcc2b795029 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.115882 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08636a97-e6dd-4a98-a6b9-f4f79ec6b9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.116533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08636a97-e6dd-4a98-a6b9-f4f79ec6b9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.116817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08636a97-e6dd-4a98-a6b9-f4f79ec6b9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.117094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08636a97-e6dd-4a98-a6b9-f4f79ec6b9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.117814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08636a97-e6dd-4a98-a6b9-f4f79ec6b9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.119505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08636a97-e6dd-4a98-a6b9-f4f79ec6b9b7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.130258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1260/2504] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:05.133899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef7fb4-93ba-4b68-8e17-bc522112303a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.143686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef7fb4-93ba-4b68-8e17-bc522112303a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.148229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d2aea-2ea8-404f-821f-d58886bf333a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.148873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2aea-2ea8-404f-821f-d58886bf333a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.149156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2aea-2ea8-404f-821f-d58886bf333a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.149434 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2aea-2ea8-404f-821f-d58886bf333a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.150151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d2aea-2ea8-404f-821f-d58886bf333a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.151784 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d2aea-2ea8-404f-821f-d58886bf333a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.161464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1245/2505] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:05.165500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08636a97-e6dd-4a98-a6b9-f4f79ec6b9b7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.175902 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08636a97-e6dd-4a98-a6b9-f4f79ec6b9b7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.180481 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553ce44-7bf9-4d5b-ae78-587a33059065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.181344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553ce44-7bf9-4d5b-ae78-587a33059065 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.181648 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553ce44-7bf9-4d5b-ae78-587a33059065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.182024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553ce44-7bf9-4d5b-ae78-587a33059065 admin admin] PATH_INFO: `/v3/projects/662aab145c244b0f887eba776704c7d1/users/aaf7b17bcae548e9a207663828352347/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.197443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553ce44-7bf9-4d5b-ae78-587a33059065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=662aab145c244b0f887eba776704c7d1, user_id=aaf7b17bcae548e9a207663828352347, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.200300 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553ce44-7bf9-4d5b-ae78-587a33059065 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.213285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1261/2506] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:05 2022] PUT /identity/v3/projects/662aab145c244b0f887eba776704c7d1/users/aaf7b17bcae548e9a207663828352347/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:11:05.225012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeca1c-9539-41e8-8c9f-8f600d8c4b3c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.225304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeca1c-9539-41e8-8c9f-8f600d8c4b3c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.225582 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeca1c-9539-41e8-8c9f-8f600d8c4b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.252006 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-0eeeca1c-9539-41e8-8c9f-8f600d8c4b3c None None] MFA Rules not processed for user `aaf7b17bcae548e9a207663828352347`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:05.253234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0eeeca1c-9539-41e8-8c9f-8f600d8c4b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:05.314333 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1246/2507] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:05.318782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553ce44-7bf9-4d5b-ae78-587a33059065 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.379894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553ce44-7bf9-4d5b-ae78-587a33059065 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.384676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143005f-7bd3-446c-b94f-8d6c98ed178b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.385334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143005f-7bd3-446c-b94f-8d6c98ed178b admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.385621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143005f-7bd3-446c-b94f-8d6c98ed178b admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.385904 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143005f-7bd3-446c-b94f-8d6c98ed178b admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.386636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143005f-7bd3-446c-b94f-8d6c98ed178b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.388539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143005f-7bd3-446c-b94f-8d6c98ed178b admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.415885 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1262/2508] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:05 2022] POST /identity/v3/projects => generated 399 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:05.419723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeeca1c-9539-41e8-8c9f-8f600d8c4b3c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.431359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeeca1c-9539-41e8-8c9f-8f600d8c4b3c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.485579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab12b8-2c28-4262-b6de-8b24b02d3d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.486238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab12b8-2c28-4262-b6de-8b24b02d3d51 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.487192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab12b8-2c28-4262-b6de-8b24b02d3d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.487474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab12b8-2c28-4262-b6de-8b24b02d3d51 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.488150 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab12b8-2c28-4262-b6de-8b24b02d3d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.490071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab12b8-2c28-4262-b6de-8b24b02d3d51 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.509180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1247/2509] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:05 2022] POST /identity/v3/users => generated 306 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:11:05.513165 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143005f-7bd3-446c-b94f-8d6c98ed178b admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.523340 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143005f-7bd3-446c-b94f-8d6c98ed178b admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.527949 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173edd10-8150-4067-b854-2e10d35e7f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.528601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173edd10-8150-4067-b854-2e10d35e7f70 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.528888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173edd10-8150-4067-b854-2e10d35e7f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.529164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173edd10-8150-4067-b854-2e10d35e7f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.529876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173edd10-8150-4067-b854-2e10d35e7f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.531494 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173edd10-8150-4067-b854-2e10d35e7f70 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.542531 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1263/2510] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:05.546864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab12b8-2c28-4262-b6de-8b24b02d3d51 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.557608 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab12b8-2c28-4262-b6de-8b24b02d3d51 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.562216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541106e-700d-4da7-a998-12baf733306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.563123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541106e-700d-4da7-a998-12baf733306a admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.563403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541106e-700d-4da7-a998-12baf733306a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.563679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541106e-700d-4da7-a998-12baf733306a admin admin] PATH_INFO: `/v3/projects/9d448a93811242109e820408f3e60a20/users/a6d5eeaa5707434f8037fcff9021f1e3/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.577812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541106e-700d-4da7-a998-12baf733306a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d448a93811242109e820408f3e60a20, user_id=a6d5eeaa5707434f8037fcff9021f1e3, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.581022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e541106e-700d-4da7-a998-12baf733306a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.593731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1248/2511] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:05 2022] PUT /identity/v3/projects/9d448a93811242109e820408f3e60a20/users/a6d5eeaa5707434f8037fcff9021f1e3/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:05.598168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173edd10-8150-4067-b854-2e10d35e7f70 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.608060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173edd10-8150-4067-b854-2e10d35e7f70 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.627811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:05.665288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb27b7-2766-4219-8e25-baea59a37a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.665985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb27b7-2766-4219-8e25-baea59a37a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.666271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb27b7-2766-4219-8e25-baea59a37a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.666583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb27b7-2766-4219-8e25-baea59a37a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.667299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb27b7-2766-4219-8e25-baea59a37a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.668893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb27b7-2766-4219-8e25-baea59a37a4d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.678804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1264/2512] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:05.682853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541106e-700d-4da7-a998-12baf733306a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.688315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:05.694849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e541106e-700d-4da7-a998-12baf733306a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.699724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9a122-0426-4cf4-aa7a-0191a588c6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.700623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9a122-0426-4cf4-aa7a-0191a588c6e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.700906 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9a122-0426-4cf4-aa7a-0191a588c6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.701177 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9a122-0426-4cf4-aa7a-0191a588c6e0 admin admin] PATH_INFO: `/v3/projects/9d448a93811242109e820408f3e60a20/users/a6d5eeaa5707434f8037fcff9021f1e3/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.703261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9a122-0426-4cf4-aa7a-0191a588c6e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d448a93811242109e820408f3e60a20, user_id=a6d5eeaa5707434f8037fcff9021f1e3, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.706393 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9a122-0426-4cf4-aa7a-0191a588c6e0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.716498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1249/2513] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:05 2022] PUT /identity/v3/projects/9d448a93811242109e820408f3e60a20/users/a6d5eeaa5707434f8037fcff9021f1e3/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:05.727039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64769c-e025-41b1-b1e3-30727f095d49 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.727335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64769c-e025-41b1-b1e3-30727f095d49 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.727617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e64769c-e025-41b1-b1e3-30727f095d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.754212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4e64769c-e025-41b1-b1e3-30727f095d49 None None] MFA Rules not processed for user `a6d5eeaa5707434f8037fcff9021f1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:05.755433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4e64769c-e025-41b1-b1e3-30727f095d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:05.820850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1265/2514] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:05 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:05.831600 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9a122-0426-4cf4-aa7a-0191a588c6e0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.833297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-87c9a122-0426-4cf4-aa7a-0191a588c6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:05.889836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9a122-0426-4cf4-aa7a-0191a588c6e0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:05.893502 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfef86e-e984-46b7-a899-bf9da52b30e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:05.894137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfef86e-e984-46b7-a899-bf9da52b30e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:05.894423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfef86e-e984-46b7-a899-bf9da52b30e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:05.894740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfef86e-e984-46b7-a899-bf9da52b30e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:05.903642 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfef86e-e984-46b7-a899-bf9da52b30e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:05.905092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfef86e-e984-46b7-a899-bf9da52b30e5 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:05.914726 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1250/2515] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:05 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:05.989722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64769c-e025-41b1-b1e3-30727f095d49 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:05.999550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e64769c-e025-41b1-b1e3-30727f095d49 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:06.003080 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1adeed-6247-4e41-bb02-cbd61c13af3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:06.003712 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1adeed-6247-4e41-bb02-cbd61c13af3b service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:06.003994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1adeed-6247-4e41-bb02-cbd61c13af3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:06.004285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1adeed-6247-4e41-bb02-cbd61c13af3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:06.060438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1adeed-6247-4e41-bb02-cbd61c13af3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:06.061864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1adeed-6247-4e41-bb02-cbd61c13af3b service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:06.070818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1266/2516] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:10.086904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfef86e-e984-46b7-a899-bf9da52b30e5 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:10.149318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfef86e-e984-46b7-a899-bf9da52b30e5 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:10.153888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:10.154785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:10.155075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:10.155357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] PATH_INFO: `/v3/users/aaf7b17bcae548e9a207663828352347` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:10.156544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf7b17bcae548e9a207663828352347)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:10.158525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:10.207940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf7b17bcae548e9a207663828352347'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.208535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf7b17bcae548e9a207663828352347'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.215632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf7b17bcae548e9a207663828352347'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.216261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf7b17bcae548e9a207663828352347'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.227933 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf7b17bcae548e9a207663828352347'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.240649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1251/2517] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:10 2022] DELETE /identity/v3/users/aaf7b17bcae548e9a207663828352347 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:11:10.245133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7c3a0-4ddc-424c-a083-f6afba6ca17a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:10.245612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7c3a0-4ddc-424c-a083-f6afba6ca17a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:10.245711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7c3a0-4ddc-424c-a083-f6afba6ca17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:10.263060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6cd7c3a0-4ddc-424c-a083-f6afba6ca17a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:10.264271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6cd7c3a0-4ddc-424c-a083-f6afba6ca17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:10.303915 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:10.323488 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1267/2518] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:11:10.332904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:10.334303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:10.347805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:10.393601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effeed77-63de-4c03-a2e8-53a2e9b8f502 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:10.397299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5df78e-f385-4983-99c4-2c37dc82ee98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:10.397938 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5df78e-f385-4983-99c4-2c37dc82ee98 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:10.398228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5df78e-f385-4983-99c4-2c37dc82ee98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:10.398537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5df78e-f385-4983-99c4-2c37dc82ee98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:10.407463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5df78e-f385-4983-99c4-2c37dc82ee98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:10.408904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5df78e-f385-4983-99c4-2c37dc82ee98 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:10.418923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1252/2519] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:10.707238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7c3a0-4ddc-424c-a083-f6afba6ca17a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:10.708692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6cd7c3a0-4ddc-424c-a083-f6afba6ca17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:10.722025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7c3a0-4ddc-424c-a083-f6afba6ca17a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:10.727161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:10.728023 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:10.728828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:10.729118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] PATH_INFO: `/v3/projects/662aab145c244b0f887eba776704c7d1` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:10.730630 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=662aab145c244b0f887eba776704c7d1)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:10.732444 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:10.778003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '662aab145c244b0f887eba776704c7d1', 'request_id': 'req-91d7a10a-6167-4531-88bd-0ae5a183dd31'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.784043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '662aab145c244b0f887eba776704c7d1', 'request_id': 'req-91d7a10a-6167-4531-88bd-0ae5a183dd31'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.789996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '662aab145c244b0f887eba776704c7d1', 'request_id': 'req-91d7a10a-6167-4531-88bd-0ae5a183dd31'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.802747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] The token cache is being invalidate because project 662aab145c244b0f887eba776704c7d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:10.803036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.803668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.805239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1268/2520] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:10 2022] DELETE /identity/v3/projects/662aab145c244b0f887eba776704c7d1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:10.809250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5df78e-f385-4983-99c4-2c37dc82ee98 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:10.819111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5df78e-f385-4983-99c4-2c37dc82ee98 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:10.875508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:10.876357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:10.876633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:10.876901 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] PATH_INFO: `/v3/users/a6d5eeaa5707434f8037fcff9021f1e3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:10.878065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6d5eeaa5707434f8037fcff9021f1e3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:10.880058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:10.930725 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6d5eeaa5707434f8037fcff9021f1e3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.931383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6d5eeaa5707434f8037fcff9021f1e3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.939201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6d5eeaa5707434f8037fcff9021f1e3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.940268 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6d5eeaa5707434f8037fcff9021f1e3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.957865 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-086f0f67-7d67-45a3-a298-e756486c9aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6d5eeaa5707434f8037fcff9021f1e3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:10.976166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1253/2521] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:10 2022] DELETE /identity/v3/users/a6d5eeaa5707434f8037fcff9021f1e3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:11.242220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.243683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:11.261158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7a10a-6167-4531-88bd-0ae5a183dd31 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.320413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.321270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.321554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.321829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] PATH_INFO: `/v3/projects/9d448a93811242109e820408f3e60a20` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.323019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d448a93811242109e820408f3e60a20)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.324922 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:11.373213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d448a93811242109e820408f3e60a20', 'request_id': 'req-af2bc42b-bac8-4604-ac14-193777d134a0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:11.380235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d448a93811242109e820408f3e60a20', 'request_id': 'req-af2bc42b-bac8-4604-ac14-193777d134a0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:11.385787 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d448a93811242109e820408f3e60a20', 'request_id': 'req-af2bc42b-bac8-4604-ac14-193777d134a0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:11.395497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:11.400651 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] The token cache is being invalidate because project 9d448a93811242109e820408f3e60a20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:11.401157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:11.402195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-af2bc42b-bac8-4604-ac14-193777d134a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:11.403867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1269/2522] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:11 2022] DELETE /identity/v3/projects/9d448a93811242109e820408f3e60a20 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:11.409055 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b57238-8d52-4240-851e-5ff17bfdb799 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.409538 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b57238-8d52-4240-851e-5ff17bfdb799 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.409645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b57238-8d52-4240-851e-5ff17bfdb799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.426800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d2b57238-8d52-4240-851e-5ff17bfdb799 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:11.427989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d2b57238-8d52-4240-851e-5ff17bfdb799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:11.457343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:11.487544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1254/2523] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:11.496217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e82a-7366-49d6-9878-265794263996 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.496511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e82a-7366-49d6-9878-265794263996 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.496788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10e82a-7366-49d6-9878-265794263996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.515279 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-1f10e82a-7366-49d6-9878-265794263996 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:11.516480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1f10e82a-7366-49d6-9878-265794263996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:11.524945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1270/2524] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:11.528956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b57238-8d52-4240-851e-5ff17bfdb799 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.538531 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b57238-8d52-4240-851e-5ff17bfdb799 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.543060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732af88-fc9e-479b-b068-05fd7e996fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.543702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732af88-fc9e-479b-b068-05fd7e996fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.543987 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732af88-fc9e-479b-b068-05fd7e996fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.544267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732af88-fc9e-479b-b068-05fd7e996fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.544986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732af88-fc9e-479b-b068-05fd7e996fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:11.545296 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732af88-fc9e-479b-b068-05fd7e996fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.546945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f732af88-fc9e-479b-b068-05fd7e996fd7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:11.558644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1255/2525] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:11.562811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10e82a-7366-49d6-9878-265794263996 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.572564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10e82a-7366-49d6-9878-265794263996 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.577118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24164731-4c34-4651-ac26-bfefa6bb1473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.577757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24164731-4c34-4651-ac26-bfefa6bb1473 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.578036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24164731-4c34-4651-ac26-bfefa6bb1473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.578310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24164731-4c34-4651-ac26-bfefa6bb1473 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.579031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24164731-4c34-4651-ac26-bfefa6bb1473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.580909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24164731-4c34-4651-ac26-bfefa6bb1473 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:11.606196 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1271/2526] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:11 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:11.610358 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732af88-fc9e-479b-b068-05fd7e996fd7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.621197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f732af88-fc9e-479b-b068-05fd7e996fd7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.675829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b5a41-7fe7-42dc-b0a2-f9a9e8f78404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.676499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b5a41-7fe7-42dc-b0a2-f9a9e8f78404 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.676810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b5a41-7fe7-42dc-b0a2-f9a9e8f78404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.677053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b5a41-7fe7-42dc-b0a2-f9a9e8f78404 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.677743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b5a41-7fe7-42dc-b0a2-f9a9e8f78404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.679955 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b5a41-7fe7-42dc-b0a2-f9a9e8f78404 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:11.700417 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1256/2527] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:11 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:11.704607 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24164731-4c34-4651-ac26-bfefa6bb1473 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.714715 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24164731-4c34-4651-ac26-bfefa6bb1473 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.719299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ce9d-de0e-4e79-bdfd-048f53ae1c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.719946 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ce9d-de0e-4e79-bdfd-048f53ae1c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.720231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ce9d-de0e-4e79-bdfd-048f53ae1c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.720506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878ce9d-de0e-4e79-bdfd-048f53ae1c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.721218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878ce9d-de0e-4e79-bdfd-048f53ae1c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.722836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878ce9d-de0e-4e79-bdfd-048f53ae1c90 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:11.732890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1272/2528] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:11.736625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b5a41-7fe7-42dc-b0a2-f9a9e8f78404 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.747347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b5a41-7fe7-42dc-b0a2-f9a9e8f78404 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.751806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a9538-a2fc-417a-a4e9-6c371e897110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.752468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a9538-a2fc-417a-a4e9-6c371e897110 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.752749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a9538-a2fc-417a-a4e9-6c371e897110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.753023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a9538-a2fc-417a-a4e9-6c371e897110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.754347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a9538-a2fc-417a-a4e9-6c371e897110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.755999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a9538-a2fc-417a-a4e9-6c371e897110 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:11.765793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1257/2529] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:11.769540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ce9d-de0e-4e79-bdfd-048f53ae1c90 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.779498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878ce9d-de0e-4e79-bdfd-048f53ae1c90 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.784101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc8134-64c8-4bb6-99c4-ec6807990776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.785057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc8134-64c8-4bb6-99c4-ec6807990776 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.785361 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc8134-64c8-4bb6-99c4-ec6807990776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.785637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc8134-64c8-4bb6-99c4-ec6807990776 admin admin] PATH_INFO: `/v3/projects/aa4e4246a3e74fdea8ee29653aa9c59e/users/437a78f90e324a6aa36f472c4136dc37/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.799974 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc8134-64c8-4bb6-99c4-ec6807990776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4e4246a3e74fdea8ee29653aa9c59e, user_id=437a78f90e324a6aa36f472c4136dc37, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.803257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc8134-64c8-4bb6-99c4-ec6807990776 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:11.816900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1273/2530] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:11 2022] PUT /identity/v3/projects/aa4e4246a3e74fdea8ee29653aa9c59e/users/437a78f90e324a6aa36f472c4136dc37/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:11:11.826890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c566340-5a78-4519-935f-a97bd153cfbc None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.827184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c566340-5a78-4519-935f-a97bd153cfbc None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.827464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c566340-5a78-4519-935f-a97bd153cfbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.853365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1c566340-5a78-4519-935f-a97bd153cfbc None None] MFA Rules not processed for user `437a78f90e324a6aa36f472c4136dc37`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:11.854617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1c566340-5a78-4519-935f-a97bd153cfbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:11.917360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1258/2531] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:11.921723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc8134-64c8-4bb6-99c4-ec6807990776 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.981334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc8134-64c8-4bb6-99c4-ec6807990776 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.986569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a733fa-45af-4c3a-ac8e-9699874f4758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.987236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a733fa-45af-4c3a-ac8e-9699874f4758 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.987522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a733fa-45af-4c3a-ac8e-9699874f4758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.987892 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a733fa-45af-4c3a-ac8e-9699874f4758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.989066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a733fa-45af-4c3a-ac8e-9699874f4758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.991255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a733fa-45af-4c3a-ac8e-9699874f4758 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.016244 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1274/2532] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:11 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:12.020791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c566340-5a78-4519-935f-a97bd153cfbc None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.031303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c566340-5a78-4519-935f-a97bd153cfbc None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.056869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:12.089319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b74538-a712-4a3f-99a3-5a40aff5e6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.090049 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b74538-a712-4a3f-99a3-5a40aff5e6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.090331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b74538-a712-4a3f-99a3-5a40aff5e6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.090641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b74538-a712-4a3f-99a3-5a40aff5e6e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.091337 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b74538-a712-4a3f-99a3-5a40aff5e6e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.093301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b74538-a712-4a3f-99a3-5a40aff5e6e5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.114937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1259/2533] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:12 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:12.119196 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a733fa-45af-4c3a-ac8e-9699874f4758 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.130100 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a733fa-45af-4c3a-ac8e-9699874f4758 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.134717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedec75e-c09f-4c67-a919-b6d442b9e951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.135372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedec75e-c09f-4c67-a919-b6d442b9e951 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.135658 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedec75e-c09f-4c67-a919-b6d442b9e951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.135938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedec75e-c09f-4c67-a919-b6d442b9e951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.136688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedec75e-c09f-4c67-a919-b6d442b9e951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.138286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedec75e-c09f-4c67-a919-b6d442b9e951 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.149318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1275/2534] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:12.153470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b74538-a712-4a3f-99a3-5a40aff5e6e5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.163482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b74538-a712-4a3f-99a3-5a40aff5e6e5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.168112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268cc1d-794a-444a-b7b6-d3e2788477fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.168977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268cc1d-794a-444a-b7b6-d3e2788477fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.169257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268cc1d-794a-444a-b7b6-d3e2788477fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.169530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268cc1d-794a-444a-b7b6-d3e2788477fe admin admin] PATH_INFO: `/v3/projects/d8f0f4de50904f1d8f4d51d458bc58f0/users/614daef011464c0fa27cac07d81ead3a/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.183920 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268cc1d-794a-444a-b7b6-d3e2788477fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f0f4de50904f1d8f4d51d458bc58f0, user_id=614daef011464c0fa27cac07d81ead3a, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.186653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268cc1d-794a-444a-b7b6-d3e2788477fe admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.200306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1260/2535] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:12 2022] PUT /identity/v3/projects/d8f0f4de50904f1d8f4d51d458bc58f0/users/614daef011464c0fa27cac07d81ead3a/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:12.204309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedec75e-c09f-4c67-a919-b6d442b9e951 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.214751 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedec75e-c09f-4c67-a919-b6d442b9e951 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.270071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515febde-11cc-41df-8bd7-50e0347065a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.270726 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515febde-11cc-41df-8bd7-50e0347065a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.270953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515febde-11cc-41df-8bd7-50e0347065a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.271230 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515febde-11cc-41df-8bd7-50e0347065a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.271942 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515febde-11cc-41df-8bd7-50e0347065a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.273540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515febde-11cc-41df-8bd7-50e0347065a9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.283080 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1276/2536] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:12.287280 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268cc1d-794a-444a-b7b6-d3e2788477fe admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.301619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268cc1d-794a-444a-b7b6-d3e2788477fe admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.305484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fdb5b-ac4f-429d-a35b-b5e605e5ee44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.306460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fdb5b-ac4f-429d-a35b-b5e605e5ee44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.306789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fdb5b-ac4f-429d-a35b-b5e605e5ee44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.307308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71fdb5b-ac4f-429d-a35b-b5e605e5ee44 admin admin] PATH_INFO: `/v3/projects/d8f0f4de50904f1d8f4d51d458bc58f0/users/614daef011464c0fa27cac07d81ead3a/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.310260 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fdb5b-ac4f-429d-a35b-b5e605e5ee44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f0f4de50904f1d8f4d51d458bc58f0, user_id=614daef011464c0fa27cac07d81ead3a, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.313003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71fdb5b-ac4f-429d-a35b-b5e605e5ee44 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.324300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1261/2537] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:12 2022] PUT /identity/v3/projects/d8f0f4de50904f1d8f4d51d458bc58f0/users/614daef011464c0fa27cac07d81ead3a/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:12.334543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea26752-74cd-435d-bd3c-d3199b38bc5f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.334837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea26752-74cd-435d-bd3c-d3199b38bc5f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.335218 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea26752-74cd-435d-bd3c-d3199b38bc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.341912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:12.362169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-dea26752-74cd-435d-bd3c-d3199b38bc5f None None] MFA Rules not processed for user `614daef011464c0fa27cac07d81ead3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:12.363916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dea26752-74cd-435d-bd3c-d3199b38bc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:12.431727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1277/2538] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:12.443025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fdb5b-ac4f-429d-a35b-b5e605e5ee44 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.444827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d71fdb5b-ac4f-429d-a35b-b5e605e5ee44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:12.500916 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71fdb5b-ac4f-429d-a35b-b5e605e5ee44 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.505370 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612448f-3ecb-40ef-9e45-67e3a744e911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.506002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612448f-3ecb-40ef-9e45-67e3a744e911 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.506288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612448f-3ecb-40ef-9e45-67e3a744e911 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.506599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612448f-3ecb-40ef-9e45-67e3a744e911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.515973 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612448f-3ecb-40ef-9e45-67e3a744e911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.517410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612448f-3ecb-40ef-9e45-67e3a744e911 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.527550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1262/2539] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:12.631010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea26752-74cd-435d-bd3c-d3199b38bc5f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.641589 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea26752-74cd-435d-bd3c-d3199b38bc5f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.645347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6acd6-5ca7-4803-b248-84db2aa3c509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.645991 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6acd6-5ca7-4803-b248-84db2aa3c509 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.646275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6acd6-5ca7-4803-b248-84db2aa3c509 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.646604 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6acd6-5ca7-4803-b248-84db2aa3c509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.703011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6acd6-5ca7-4803-b248-84db2aa3c509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.704537 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6acd6-5ca7-4803-b248-84db2aa3c509 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.713468 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1278/2540] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:15.259947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612448f-3ecb-40ef-9e45-67e3a744e911 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:15.321899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612448f-3ecb-40ef-9e45-67e3a744e911 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:15.326392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:15.327279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:15.327563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:15.327840 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] PATH_INFO: `/v3/users/437a78f90e324a6aa36f472c4136dc37` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:15.329019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=437a78f90e324a6aa36f472c4136dc37)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:15.331013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:15.357355 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:15.376860 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437a78f90e324a6aa36f472c4136dc37'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:15.377488 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437a78f90e324a6aa36f472c4136dc37'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:15.384929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437a78f90e324a6aa36f472c4136dc37'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:15.385536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437a78f90e324a6aa36f472c4136dc37'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:15.397014 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437a78f90e324a6aa36f472c4136dc37'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:15.408750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1263/2541] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:15 2022] DELETE /identity/v3/users/437a78f90e324a6aa36f472c4136dc37 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:15.413374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9bdb9-995b-49b5-9f11-d435db6e5316 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:15.413832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9bdb9-995b-49b5-9f11-d435db6e5316 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:15.413948 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9bdb9-995b-49b5-9f11-d435db6e5316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:15.431042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-dea9bdb9-995b-49b5-9f11-d435db6e5316 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:15.432237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dea9bdb9-995b-49b5-9f11-d435db6e5316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:15.489564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1279/2542] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:15.499166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:15.500536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:15.562752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc3b15-f174-4f14-9119-0e54fa3c4f6d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:15.566890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53115998-8f77-4a47-adf5-5331ee1bfb32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:15.567579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53115998-8f77-4a47-adf5-5331ee1bfb32 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:15.567859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53115998-8f77-4a47-adf5-5331ee1bfb32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:15.568135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53115998-8f77-4a47-adf5-5331ee1bfb32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:15.577679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53115998-8f77-4a47-adf5-5331ee1bfb32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:15.579197 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53115998-8f77-4a47-adf5-5331ee1bfb32 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:15.588942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1264/2543] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:15.856331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9bdb9-995b-49b5-9f11-d435db6e5316 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:15.857813 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-dea9bdb9-995b-49b5-9f11-d435db6e5316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:15.870642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9bdb9-995b-49b5-9f11-d435db6e5316 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:15.875291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:15.876154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:15.876434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:15.876709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] PATH_INFO: `/v3/projects/aa4e4246a3e74fdea8ee29653aa9c59e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:15.877880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4e4246a3e74fdea8ee29653aa9c59e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:15.879884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:15.904424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:15.926177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4e4246a3e74fdea8ee29653aa9c59e', 'request_id': 'req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:15.932197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4e4246a3e74fdea8ee29653aa9c59e', 'request_id': 'req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:15.937704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4e4246a3e74fdea8ee29653aa9c59e', 'request_id': 'req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:15.949976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] The token cache is being invalidate because project aa4e4246a3e74fdea8ee29653aa9c59e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:15.950262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:15.951412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:15.952471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1280/2544] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:15 2022] DELETE /identity/v3/projects/aa4e4246a3e74fdea8ee29653aa9c59e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:15.956697 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53115998-8f77-4a47-adf5-5331ee1bfb32 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:15.967743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53115998-8f77-4a47-adf5-5331ee1bfb32 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.028942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.029735 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.030021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.030297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] PATH_INFO: `/v3/users/614daef011464c0fa27cac07d81ead3a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.031546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=614daef011464c0fa27cac07d81ead3a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.033481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.077873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '614daef011464c0fa27cac07d81ead3a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:16.078545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '614daef011464c0fa27cac07d81ead3a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:16.080173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:16.087777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '614daef011464c0fa27cac07d81ead3a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:16.089092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '614daef011464c0fa27cac07d81ead3a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:16.101759 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-afc52d4c-68bc-4f97-b994-ce0addfcc779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '614daef011464c0fa27cac07d81ead3a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:16.114237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1265/2545] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:15 2022] DELETE /identity/v3/users/614daef011464c0fa27cac07d81ead3a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:16.376843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.378698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:16.405803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1211b-6ad3-4651-aa9c-b9a18c151e16 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.466672 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.467728 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.468025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.468306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] PATH_INFO: `/v3/projects/d8f0f4de50904f1d8f4d51d458bc58f0` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.469504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f0f4de50904f1d8f4d51d458bc58f0)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.471454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.520031 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f0f4de50904f1d8f4d51d458bc58f0', 'request_id': 'req-12eb83b8-a558-496c-8f04-670fb15cf60c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:16.526308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f0f4de50904f1d8f4d51d458bc58f0', 'request_id': 'req-12eb83b8-a558-496c-8f04-670fb15cf60c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:16.531963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f0f4de50904f1d8f4d51d458bc58f0', 'request_id': 'req-12eb83b8-a558-496c-8f04-670fb15cf60c'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:16.544873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] The token cache is being invalidate because project d8f0f4de50904f1d8f4d51d458bc58f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:16.545237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:16.545864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-12eb83b8-a558-496c-8f04-670fb15cf60c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:16.547444 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1281/2546] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:16 2022] DELETE /identity/v3/projects/d8f0f4de50904f1d8f4d51d458bc58f0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:16.554644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2d67f-a2c0-437f-a761-a77ee49c8958 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.554644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2d67f-a2c0-437f-a761-a77ee49c8958 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.554644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2d67f-a2c0-437f-a761-a77ee49c8958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.572509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-8ee2d67f-a2c0-437f-a761-a77ee49c8958 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:16.573955 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8ee2d67f-a2c0-437f-a761-a77ee49c8958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:16.634514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1266/2547] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:16.643344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb6744-13d7-4f1f-b040-20c7a555c6ba None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.643629 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb6744-13d7-4f1f-b040-20c7a555c6ba None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.643905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb6744-13d7-4f1f-b040-20c7a555c6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.662718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d7bb6744-13d7-4f1f-b040-20c7a555c6ba None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:16.664074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d7bb6744-13d7-4f1f-b040-20c7a555c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:16.672282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1282/2548] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:16.677128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2d67f-a2c0-437f-a761-a77ee49c8958 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.687349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2d67f-a2c0-437f-a761-a77ee49c8958 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.691934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a354b-998b-4251-b3ca-72911824e7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.692579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a354b-998b-4251-b3ca-72911824e7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.692863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a354b-998b-4251-b3ca-72911824e7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.693143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a354b-998b-4251-b3ca-72911824e7d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.693863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a354b-998b-4251-b3ca-72911824e7d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:16.694172 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a354b-998b-4251-b3ca-72911824e7d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.695828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a354b-998b-4251-b3ca-72911824e7d9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.707467 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1267/2549] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:16.711688 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb6744-13d7-4f1f-b040-20c7a555c6ba None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.720898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb6744-13d7-4f1f-b040-20c7a555c6ba None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.726014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdb927-ab36-47e0-84a7-267d109e25ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.726692 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb927-ab36-47e0-84a7-267d109e25ef admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.726980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb927-ab36-47e0-84a7-267d109e25ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.727259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb927-ab36-47e0-84a7-267d109e25ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.727945 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdb927-ab36-47e0-84a7-267d109e25ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.729844 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdb927-ab36-47e0-84a7-267d109e25ef admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.755370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1283/2550] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:16 2022] POST /identity/v3/projects => generated 389 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:16.759320 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a354b-998b-4251-b3ca-72911824e7d9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.769175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a354b-998b-4251-b3ca-72911824e7d9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.825545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4957-144b-4035-ac41-3f4dc93a815f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.826206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4957-144b-4035-ac41-3f4dc93a815f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.826518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4957-144b-4035-ac41-3f4dc93a815f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.826800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b4957-144b-4035-ac41-3f4dc93a815f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.827489 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b4957-144b-4035-ac41-3f4dc93a815f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.829421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b4957-144b-4035-ac41-3f4dc93a815f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.849536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1268/2551] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:16 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:16.853615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdb927-ab36-47e0-84a7-267d109e25ef admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.864016 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdb927-ab36-47e0-84a7-267d109e25ef admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.868920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6615393-69e7-4214-8c88-1f8271599405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.869576 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6615393-69e7-4214-8c88-1f8271599405 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.869862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6615393-69e7-4214-8c88-1f8271599405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.870174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6615393-69e7-4214-8c88-1f8271599405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.870943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6615393-69e7-4214-8c88-1f8271599405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.872551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6615393-69e7-4214-8c88-1f8271599405 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.882821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1284/2552] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:16.886333 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4957-144b-4035-ac41-3f4dc93a815f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.896554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b4957-144b-4035-ac41-3f4dc93a815f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.901130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964589b-aeba-48c3-933c-a926e260d314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.901774 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964589b-aeba-48c3-933c-a926e260d314 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.902058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964589b-aeba-48c3-933c-a926e260d314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.902336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964589b-aeba-48c3-933c-a926e260d314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.903086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964589b-aeba-48c3-933c-a926e260d314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.904757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964589b-aeba-48c3-933c-a926e260d314 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.915130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1269/2553] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:16.919027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6615393-69e7-4214-8c88-1f8271599405 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.924065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:16.930008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6615393-69e7-4214-8c88-1f8271599405 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.935467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f9a2-46d2-4f43-ba18-c2ceb1b5b0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.936363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f9a2-46d2-4f43-ba18-c2ceb1b5b0a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.936644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f9a2-46d2-4f43-ba18-c2ceb1b5b0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.936918 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f9a2-46d2-4f43-ba18-c2ceb1b5b0a3 admin admin] PATH_INFO: `/v3/projects/a01e1d80916342d0bb681acec41a896d/users/75703635a5264bab9943b93eef6eecea/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.951096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97f9a2-46d2-4f43-ba18-c2ceb1b5b0a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a01e1d80916342d0bb681acec41a896d, user_id=75703635a5264bab9943b93eef6eecea, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.953744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97f9a2-46d2-4f43-ba18-c2ceb1b5b0a3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.967166 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1285/2554] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:16 2022] PUT /identity/v3/projects/a01e1d80916342d0bb681acec41a896d/users/75703635a5264bab9943b93eef6eecea/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:16.977747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56541c-1096-4603-9439-9b5027c98e6c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.978044 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56541c-1096-4603-9439-9b5027c98e6c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.978336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56541c-1096-4603-9439-9b5027c98e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.004536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9e56541c-1096-4603-9439-9b5027c98e6c None None] MFA Rules not processed for user `75703635a5264bab9943b93eef6eecea`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:17.005741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9e56541c-1096-4603-9439-9b5027c98e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:17.024309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:17.071557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1270/2555] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:17.075847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f9a2-46d2-4f43-ba18-c2ceb1b5b0a3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.135994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f9a2-46d2-4f43-ba18-c2ceb1b5b0a3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.140605 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f0564-03ce-42b1-9dfd-96420cdcc485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.141257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f0564-03ce-42b1-9dfd-96420cdcc485 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.141546 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f0564-03ce-42b1-9dfd-96420cdcc485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.141828 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f0564-03ce-42b1-9dfd-96420cdcc485 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.142540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f0564-03ce-42b1-9dfd-96420cdcc485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.144928 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f0564-03ce-42b1-9dfd-96420cdcc485 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.169625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1286/2556] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:17 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:17.173795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56541c-1096-4603-9439-9b5027c98e6c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.184445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56541c-1096-4603-9439-9b5027c98e6c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.239312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d055b-21f4-4f13-a1f4-25cf484940be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.240212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d055b-21f4-4f13-a1f4-25cf484940be admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.240503 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d055b-21f4-4f13-a1f4-25cf484940be admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.240777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d055b-21f4-4f13-a1f4-25cf484940be admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.241447 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d055b-21f4-4f13-a1f4-25cf484940be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.243421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d055b-21f4-4f13-a1f4-25cf484940be admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.263035 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1271/2557] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:17 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:17.267176 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f0564-03ce-42b1-9dfd-96420cdcc485 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.277821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f0564-03ce-42b1-9dfd-96420cdcc485 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.282342 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792de1ae-d4ab-4967-ad5e-7a97da0ba523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.283018 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792de1ae-d4ab-4967-ad5e-7a97da0ba523 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.283302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792de1ae-d4ab-4967-ad5e-7a97da0ba523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.283580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792de1ae-d4ab-4967-ad5e-7a97da0ba523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.284288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792de1ae-d4ab-4967-ad5e-7a97da0ba523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.285859 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792de1ae-d4ab-4967-ad5e-7a97da0ba523 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.296090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1287/2558] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:17.300057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d055b-21f4-4f13-a1f4-25cf484940be admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.310113 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d055b-21f4-4f13-a1f4-25cf484940be admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.314649 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4ce7e-8768-48fd-8ea8-776f3a947633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.315507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4ce7e-8768-48fd-8ea8-776f3a947633 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.315785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4ce7e-8768-48fd-8ea8-776f3a947633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.316058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4ce7e-8768-48fd-8ea8-776f3a947633 admin admin] PATH_INFO: `/v3/projects/ae7c00930b9c49579a942d954af4f582/users/88a8f958dfe142cca49bae76e2c91048/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.330791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4ce7e-8768-48fd-8ea8-776f3a947633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7c00930b9c49579a942d954af4f582, user_id=88a8f958dfe142cca49bae76e2c91048, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.334054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4ce7e-8768-48fd-8ea8-776f3a947633 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.347169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1272/2559] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:17 2022] PUT /identity/v3/projects/ae7c00930b9c49579a942d954af4f582/users/88a8f958dfe142cca49bae76e2c91048/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:11:17.350939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792de1ae-d4ab-4967-ad5e-7a97da0ba523 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.360690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792de1ae-d4ab-4967-ad5e-7a97da0ba523 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.415628 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9404c224-77dc-4e20-bf10-9aa1f945eda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.416278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404c224-77dc-4e20-bf10-9aa1f945eda4 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.416560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404c224-77dc-4e20-bf10-9aa1f945eda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.416836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9404c224-77dc-4e20-bf10-9aa1f945eda4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.417539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404c224-77dc-4e20-bf10-9aa1f945eda4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.419146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9404c224-77dc-4e20-bf10-9aa1f945eda4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.428698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1288/2560] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:17.432710 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4ce7e-8768-48fd-8ea8-776f3a947633 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.443435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4ce7e-8768-48fd-8ea8-776f3a947633 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.447976 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61a6bd-3c23-4713-a8a7-de61ba4c8154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.448842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61a6bd-3c23-4713-a8a7-de61ba4c8154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.449121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61a6bd-3c23-4713-a8a7-de61ba4c8154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.449393 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61a6bd-3c23-4713-a8a7-de61ba4c8154 admin admin] PATH_INFO: `/v3/projects/ae7c00930b9c49579a942d954af4f582/users/88a8f958dfe142cca49bae76e2c91048/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.451386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61a6bd-3c23-4713-a8a7-de61ba4c8154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7c00930b9c49579a942d954af4f582, user_id=88a8f958dfe142cca49bae76e2c91048, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.454025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61a6bd-3c23-4713-a8a7-de61ba4c8154 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.464639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1273/2561] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:17 2022] PUT /identity/v3/projects/ae7c00930b9c49579a942d954af4f582/users/88a8f958dfe142cca49bae76e2c91048/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:11:17.475279 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee45628-7d9b-4b1b-af2a-842f1d7f5423 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.475753 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee45628-7d9b-4b1b-af2a-842f1d7f5423 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.475897 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee45628-7d9b-4b1b-af2a-842f1d7f5423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.501129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-4ee45628-7d9b-4b1b-af2a-842f1d7f5423 None None] MFA Rules not processed for user `88a8f958dfe142cca49bae76e2c91048`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:17.502312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4ee45628-7d9b-4b1b-af2a-842f1d7f5423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:17.539413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:17.570071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1289/2562] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:17.580872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61a6bd-3c23-4713-a8a7-de61ba4c8154 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.582671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3f61a6bd-3c23-4713-a8a7-de61ba4c8154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:17.648526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61a6bd-3c23-4713-a8a7-de61ba4c8154 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.652170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4856d4-02b1-4a23-a9db-31c80f8e967a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.652811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4856d4-02b1-4a23-a9db-31c80f8e967a service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.653093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4856d4-02b1-4a23-a9db-31c80f8e967a service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.653371 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4856d4-02b1-4a23-a9db-31c80f8e967a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.662242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4856d4-02b1-4a23-a9db-31c80f8e967a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.663744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4856d4-02b1-4a23-a9db-31c80f8e967a service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.673462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1274/2563] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:17.796060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee45628-7d9b-4b1b-af2a-842f1d7f5423 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.857269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee45628-7d9b-4b1b-af2a-842f1d7f5423 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.861596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.862469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.862764 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.863565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] PATH_INFO: `/v3/users/75703635a5264bab9943b93eef6eecea` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.864734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75703635a5264bab9943b93eef6eecea)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.866951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.911565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75703635a5264bab9943b93eef6eecea'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:17.923336 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75703635a5264bab9943b93eef6eecea'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:17.931487 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75703635a5264bab9943b93eef6eecea'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:17.932158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75703635a5264bab9943b93eef6eecea'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:17.939338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75703635a5264bab9943b93eef6eecea'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:17.945223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1290/2564] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:17 2022] DELETE /identity/v3/users/75703635a5264bab9943b93eef6eecea => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:17.949727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0da73-b282-49b1-95ee-b184e306190d None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.950020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0da73-b282-49b1-95ee-b184e306190d None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.950305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0da73-b282-49b1-95ee-b184e306190d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.968286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-74b0da73-b282-49b1-95ee-b184e306190d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:17.969485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-74b0da73-b282-49b1-95ee-b184e306190d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:17.979022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:18.029432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1275/2565] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:18.039405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:18.040797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:18.098024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fd735-f83b-47e3-8586-f17781a04cb4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:18.101839 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747cd36-550a-42bf-8771-8df8a0e84e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:18.102500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747cd36-550a-42bf-8771-8df8a0e84e29 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:18.102790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747cd36-550a-42bf-8771-8df8a0e84e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:18.103074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8747cd36-550a-42bf-8771-8df8a0e84e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:18.110107 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:18.112433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747cd36-550a-42bf-8771-8df8a0e84e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:18.115201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8747cd36-550a-42bf-8771-8df8a0e84e29 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:18.125923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1291/2566] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:18.410621 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0da73-b282-49b1-95ee-b184e306190d None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:18.412155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-74b0da73-b282-49b1-95ee-b184e306190d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:18.424670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0da73-b282-49b1-95ee-b184e306190d None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:18.429410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:18.430277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:18.430601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:18.430888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] PATH_INFO: `/v3/projects/a01e1d80916342d0bb681acec41a896d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:18.432087 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a01e1d80916342d0bb681acec41a896d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:18.434124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:18.480618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a01e1d80916342d0bb681acec41a896d', 'request_id': 'req-210a5a52-23ba-4f34-87d7-ba4942600151'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:18.486043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a01e1d80916342d0bb681acec41a896d', 'request_id': 'req-210a5a52-23ba-4f34-87d7-ba4942600151'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:18.492620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a01e1d80916342d0bb681acec41a896d', 'request_id': 'req-210a5a52-23ba-4f34-87d7-ba4942600151'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:18.506116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] The token cache is being invalidate because project a01e1d80916342d0bb681acec41a896d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:18.506706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:18.507354 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:18.509234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1276/2567] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:18 2022] DELETE /identity/v3/projects/a01e1d80916342d0bb681acec41a896d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:18.513744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747cd36-550a-42bf-8771-8df8a0e84e29 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:18.524081 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8747cd36-550a-42bf-8771-8df8a0e84e29 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:18.578617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:18.579539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:18.579822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:18.580097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] PATH_INFO: `/v3/users/88a8f958dfe142cca49bae76e2c91048` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:18.581270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88a8f958dfe142cca49bae76e2c91048)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:18.583733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:18.636013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88a8f958dfe142cca49bae76e2c91048'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:18.648434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88a8f958dfe142cca49bae76e2c91048'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:18.655903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88a8f958dfe142cca49bae76e2c91048'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:18.656485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88a8f958dfe142cca49bae76e2c91048'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:18.663907 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-bf2cfd27-e650-45a5-876f-0901655c75e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88a8f958dfe142cca49bae76e2c91048'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:18.670021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1292/2568] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:18 2022] DELETE /identity/v3/users/88a8f958dfe142cca49bae76e2c91048 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:18.928285 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:18.929706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:18.943713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a5a52-23ba-4f34-87d7-ba4942600151 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.001063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.001949 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.002237 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.002546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] PATH_INFO: `/v3/projects/ae7c00930b9c49579a942d954af4f582` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.004242 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae7c00930b9c49579a942d954af4f582)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.006180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.046166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:19.054678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae7c00930b9c49579a942d954af4f582', 'request_id': 'req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:19.060460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae7c00930b9c49579a942d954af4f582', 'request_id': 'req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:19.067084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae7c00930b9c49579a942d954af4f582', 'request_id': 'req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:19.078827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] The token cache is being invalidate because project ae7c00930b9c49579a942d954af4f582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:19.079116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:19.079743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c1252eb5-c532-4a43-b2e0-9b1a8b932b14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:19.081267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1277/2569] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:18 2022] DELETE /identity/v3/projects/ae7c00930b9c49579a942d954af4f582 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:19.086056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0863dd-5c1d-48b5-8f2f-c90f1c366009 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.086347 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0863dd-5c1d-48b5-8f2f-c90f1c366009 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.086655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0863dd-5c1d-48b5-8f2f-c90f1c366009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.103924 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5b0863dd-5c1d-48b5-8f2f-c90f1c366009 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:19.105130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5b0863dd-5c1d-48b5-8f2f-c90f1c366009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:19.164124 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1293/2570] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:19.172167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7700a-d30e-4f13-bd5b-3496f363747a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.172399 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7700a-d30e-4f13-bd5b-3496f363747a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.172680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7700a-d30e-4f13-bd5b-3496f363747a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.191837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-82f7700a-d30e-4f13-bd5b-3496f363747a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:19.193044 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-82f7700a-d30e-4f13-bd5b-3496f363747a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:19.201010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1278/2571] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:19.205151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0863dd-5c1d-48b5-8f2f-c90f1c366009 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.214718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0863dd-5c1d-48b5-8f2f-c90f1c366009 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.219237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0103c78-7bc7-46ed-8a1c-26997b6d857d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.219879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0103c78-7bc7-46ed-8a1c-26997b6d857d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.220160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0103c78-7bc7-46ed-8a1c-26997b6d857d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.220434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0103c78-7bc7-46ed-8a1c-26997b6d857d admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.221149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0103c78-7bc7-46ed-8a1c-26997b6d857d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:19.221485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0103c78-7bc7-46ed-8a1c-26997b6d857d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.223100 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0103c78-7bc7-46ed-8a1c-26997b6d857d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.234238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1294/2572] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:19.238829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7700a-d30e-4f13-bd5b-3496f363747a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.248299 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7700a-d30e-4f13-bd5b-3496f363747a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.252986 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5cc28-058d-46f2-9216-501666bf30ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.254256 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5cc28-058d-46f2-9216-501666bf30ae admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.254585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5cc28-058d-46f2-9216-501666bf30ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.254869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5cc28-058d-46f2-9216-501666bf30ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.255554 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5cc28-058d-46f2-9216-501666bf30ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.257464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5cc28-058d-46f2-9216-501666bf30ae admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.283357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1279/2573] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:19 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:19.287344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0103c78-7bc7-46ed-8a1c-26997b6d857d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.292382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:19.298459 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0103c78-7bc7-46ed-8a1c-26997b6d857d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.358101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452eb19-341e-446c-81e0-543bf711ec94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.358803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452eb19-341e-446c-81e0-543bf711ec94 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.359092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452eb19-341e-446c-81e0-543bf711ec94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.359372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452eb19-341e-446c-81e0-543bf711ec94 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.360073 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452eb19-341e-446c-81e0-543bf711ec94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.361992 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452eb19-341e-446c-81e0-543bf711ec94 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.382953 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1295/2574] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:19 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:19.386995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5cc28-058d-46f2-9216-501666bf30ae admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.397267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5cc28-058d-46f2-9216-501666bf30ae admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.401880 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976145ba-bb49-47bd-9259-c11961db1add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.402573 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976145ba-bb49-47bd-9259-c11961db1add admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.402862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976145ba-bb49-47bd-9259-c11961db1add admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.403144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976145ba-bb49-47bd-9259-c11961db1add admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.403869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976145ba-bb49-47bd-9259-c11961db1add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.405770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976145ba-bb49-47bd-9259-c11961db1add admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.416456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1280/2575] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:19.420138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452eb19-341e-446c-81e0-543bf711ec94 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.430661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452eb19-341e-446c-81e0-543bf711ec94 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.435216 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67f50c-f981-476e-808d-4463fc74bcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.435869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67f50c-f981-476e-808d-4463fc74bcb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.436159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67f50c-f981-476e-808d-4463fc74bcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.436538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67f50c-f981-476e-808d-4463fc74bcb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.437261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67f50c-f981-476e-808d-4463fc74bcb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.438893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67f50c-f981-476e-808d-4463fc74bcb4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.449068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1296/2576] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:19.452960 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976145ba-bb49-47bd-9259-c11961db1add admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.463072 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976145ba-bb49-47bd-9259-c11961db1add admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.467661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4cfa2-b998-44bd-9d30-75291685ed6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.468530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4cfa2-b998-44bd-9d30-75291685ed6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.468816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4cfa2-b998-44bd-9d30-75291685ed6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.469095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4cfa2-b998-44bd-9d30-75291685ed6c admin admin] PATH_INFO: `/v3/projects/4a72530d6b484390bc062bf09b056fff/users/9755a846aec345348d66100c72d433c0/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.483998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4cfa2-b998-44bd-9d30-75291685ed6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a72530d6b484390bc062bf09b056fff, user_id=9755a846aec345348d66100c72d433c0, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.486782 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4cfa2-b998-44bd-9d30-75291685ed6c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.500078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1281/2577] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:19 2022] PUT /identity/v3/projects/4a72530d6b484390bc062bf09b056fff/users/9755a846aec345348d66100c72d433c0/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:19.510395 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dc371-63e3-49d8-9ab9-71edbee48556 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.510724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dc371-63e3-49d8-9ab9-71edbee48556 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.511013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dc371-63e3-49d8-9ab9-71edbee48556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.537400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9d0dc371-63e3-49d8-9ab9-71edbee48556 None None] MFA Rules not processed for user `9755a846aec345348d66100c72d433c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:19.538640 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9d0dc371-63e3-49d8-9ab9-71edbee48556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:19.601207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1297/2578] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:19.605610 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4cfa2-b998-44bd-9d30-75291685ed6c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.669687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4cfa2-b998-44bd-9d30-75291685ed6c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.674497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c7a2c-a1a7-448d-b132-438dc18be7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.675166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c7a2c-a1a7-448d-b132-438dc18be7ab admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.675471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c7a2c-a1a7-448d-b132-438dc18be7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.676312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c7a2c-a1a7-448d-b132-438dc18be7ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.677020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c7a2c-a1a7-448d-b132-438dc18be7ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.679276 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c7a2c-a1a7-448d-b132-438dc18be7ab admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.705480 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1282/2579] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:19 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:11:19.709381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dc371-63e3-49d8-9ab9-71edbee48556 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.719893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dc371-63e3-49d8-9ab9-71edbee48556 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.775772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee23fe-f562-4320-8073-42304624074c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.776437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee23fe-f562-4320-8073-42304624074c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.776724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee23fe-f562-4320-8073-42304624074c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.777003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee23fe-f562-4320-8073-42304624074c admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.777679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee23fe-f562-4320-8073-42304624074c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.779624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee23fe-f562-4320-8073-42304624074c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.794917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:19.802925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1298/2580] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:19 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:19.804208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c7a2c-a1a7-448d-b132-438dc18be7ab admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.814871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c7a2c-a1a7-448d-b132-438dc18be7ab admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.819507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd056f4-dc31-4adc-bef6-f1d2c92bb4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.820163 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd056f4-dc31-4adc-bef6-f1d2c92bb4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.820479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd056f4-dc31-4adc-bef6-f1d2c92bb4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.820761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd056f4-dc31-4adc-bef6-f1d2c92bb4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.821494 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd056f4-dc31-4adc-bef6-f1d2c92bb4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.823148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd056f4-dc31-4adc-bef6-f1d2c92bb4a6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.834051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1283/2581] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:19.838167 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee23fe-f562-4320-8073-42304624074c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.848565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee23fe-f562-4320-8073-42304624074c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.853143 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b7df4-fd5b-46c9-921e-9f895b00bd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.854015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b7df4-fd5b-46c9-921e-9f895b00bd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.854298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b7df4-fd5b-46c9-921e-9f895b00bd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.854617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b7df4-fd5b-46c9-921e-9f895b00bd2c admin admin] PATH_INFO: `/v3/projects/7e27561d2dc240bbb0c2dde73e144349/users/8d4337ba209b4f6abd37cf3d62ea1f09/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.869104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b7df4-fd5b-46c9-921e-9f895b00bd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e27561d2dc240bbb0c2dde73e144349, user_id=8d4337ba209b4f6abd37cf3d62ea1f09, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.871800 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b7df4-fd5b-46c9-921e-9f895b00bd2c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.884515 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1299/2582] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:19 2022] PUT /identity/v3/projects/7e27561d2dc240bbb0c2dde73e144349/users/8d4337ba209b4f6abd37cf3d62ea1f09/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:11:19.888403 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd056f4-dc31-4adc-bef6-f1d2c92bb4a6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.898743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd056f4-dc31-4adc-bef6-f1d2c92bb4a6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.935691 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:19.954623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fce192-49ea-42e9-8286-a4e97ae605ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.955289 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce192-49ea-42e9-8286-a4e97ae605ac admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.955627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce192-49ea-42e9-8286-a4e97ae605ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.955856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce192-49ea-42e9-8286-a4e97ae605ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.956570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fce192-49ea-42e9-8286-a4e97ae605ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.958755 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fce192-49ea-42e9-8286-a4e97ae605ac admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:19.968272 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1284/2583] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:19.972988 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b7df4-fd5b-46c9-921e-9f895b00bd2c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.983239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b7df4-fd5b-46c9-921e-9f895b00bd2c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:19.988003 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a597db-8f88-4188-bab3-fe009e65717a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:19.988824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a597db-8f88-4188-bab3-fe009e65717a admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.989115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a597db-8f88-4188-bab3-fe009e65717a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.989394 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a597db-8f88-4188-bab3-fe009e65717a admin admin] PATH_INFO: `/v3/projects/7e27561d2dc240bbb0c2dde73e144349/users/8d4337ba209b4f6abd37cf3d62ea1f09/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.991671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a597db-8f88-4188-bab3-fe009e65717a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e27561d2dc240bbb0c2dde73e144349, user_id=8d4337ba209b4f6abd37cf3d62ea1f09, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:19.994327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a597db-8f88-4188-bab3-fe009e65717a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:20.004822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1300/2584] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:19 2022] PUT /identity/v3/projects/7e27561d2dc240bbb0c2dde73e144349/users/8d4337ba209b4f6abd37cf3d62ea1f09/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:20.015876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd9e22-67a6-40f7-85a0-2512350717c5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:20.016173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd9e22-67a6-40f7-85a0-2512350717c5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:20.016458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd9e22-67a6-40f7-85a0-2512350717c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:20.042862 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-defd9e22-67a6-40f7-85a0-2512350717c5 None None] MFA Rules not processed for user `8d4337ba209b4f6abd37cf3d62ea1f09`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:20.044041 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-defd9e22-67a6-40f7-85a0-2512350717c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:20.109306 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1285/2585] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:20.120017 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a597db-8f88-4188-bab3-fe009e65717a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:20.121710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b3a597db-8f88-4188-bab3-fe009e65717a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:20.177221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a597db-8f88-4188-bab3-fe009e65717a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:20.181103 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b87ddd-f443-4518-b091-24e110b2b802 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:20.181793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b87ddd-f443-4518-b091-24e110b2b802 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:20.182036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b87ddd-f443-4518-b091-24e110b2b802 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:20.182321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b87ddd-f443-4518-b091-24e110b2b802 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:20.193039 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b87ddd-f443-4518-b091-24e110b2b802 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:20.194500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b87ddd-f443-4518-b091-24e110b2b802 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:20.204080 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1301/2586] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:20.718183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd9e22-67a6-40f7-85a0-2512350717c5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:20.782429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd9e22-67a6-40f7-85a0-2512350717c5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:20.788160 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:20.789033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:20.789319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:20.789606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] PATH_INFO: `/v3/users/9755a846aec345348d66100c72d433c0` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:20.790952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=9755a846aec345348d66100c72d433c0)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:20.792887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:20.853420 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9755a846aec345348d66100c72d433c0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:20.854151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9755a846aec345348d66100c72d433c0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:20.862890 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9755a846aec345348d66100c72d433c0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:20.864386 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9755a846aec345348d66100c72d433c0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:20.872247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:20.878301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9755a846aec345348d66100c72d433c0'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:20.892143 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1286/2587] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:20 2022] DELETE /identity/v3/users/9755a846aec345348d66100c72d433c0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:20.896577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadc62b-b0fe-448b-9722-5c95e23de081 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:20.897097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadc62b-b0fe-448b-9722-5c95e23de081 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:20.897224 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dadc62b-b0fe-448b-9722-5c95e23de081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:20.914876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-3dadc62b-b0fe-448b-9722-5c95e23de081 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:20.916060 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3dadc62b-b0fe-448b-9722-5c95e23de081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:20.972463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1302/2588] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:20.982730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:20.984217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:21.043942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0b4f5-86cb-4b7b-8c39-2501dbe56aff admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:21.047449 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f5405-b262-46fc-b3f9-3863c5553bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:21.048083 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f5405-b262-46fc-b3f9-3863c5553bbc service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:21.048360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f5405-b262-46fc-b3f9-3863c5553bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:21.048633 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f5405-b262-46fc-b3f9-3863c5553bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:21.058033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f5405-b262-46fc-b3f9-3863c5553bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:21.059506 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f5405-b262-46fc-b3f9-3863c5553bbc service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:21.068751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1287/2589] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:21.325563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadc62b-b0fe-448b-9722-5c95e23de081 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:21.327007 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3dadc62b-b0fe-448b-9722-5c95e23de081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:21.339318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dadc62b-b0fe-448b-9722-5c95e23de081 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:21.346662 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:21.348862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:21.349296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:21.349750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] PATH_INFO: `/v3/projects/4a72530d6b484390bc062bf09b056fff` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:21.351566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a72530d6b484390bc062bf09b056fff)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:21.354084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:21.401184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a72530d6b484390bc062bf09b056fff', 'request_id': 'req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.407341 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a72530d6b484390bc062bf09b056fff', 'request_id': 'req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.412738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:21.413955 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a72530d6b484390bc062bf09b056fff', 'request_id': 'req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.426608 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] The token cache is being invalidate because project 4a72530d6b484390bc062bf09b056fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:21.426894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.427506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.429022 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1303/2590] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:21 2022] DELETE /identity/v3/projects/4a72530d6b484390bc062bf09b056fff => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:21.433023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f5405-b262-46fc-b3f9-3863c5553bbc service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:21.442798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f5405-b262-46fc-b3f9-3863c5553bbc service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:21.498174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:21.499066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:21.499366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:21.499620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] PATH_INFO: `/v3/users/8d4337ba209b4f6abd37cf3d62ea1f09` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:21.500781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4337ba209b4f6abd37cf3d62ea1f09)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:21.502738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:21.547428 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4337ba209b4f6abd37cf3d62ea1f09'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.548074 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4337ba209b4f6abd37cf3d62ea1f09'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.555507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4337ba209b4f6abd37cf3d62ea1f09'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.556118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4337ba209b4f6abd37cf3d62ea1f09'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.567674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f44a804b-a9c8-46b8-9283-c39137e05544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4337ba209b4f6abd37cf3d62ea1f09'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.579679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1288/2591] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:21 2022] DELETE /identity/v3/users/8d4337ba209b4f6abd37cf3d62ea1f09 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:21.838296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:21.839737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:21.856113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabe6f5-a2ad-4f51-927e-c1831b25cf24 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:21.917610 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:21.918492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:21.918782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:21.919048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] PATH_INFO: `/v3/projects/7e27561d2dc240bbb0c2dde73e144349` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:21.920211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e27561d2dc240bbb0c2dde73e144349)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:21.922092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:21.966383 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e27561d2dc240bbb0c2dde73e144349', 'request_id': 'req-274a3b9d-c463-44ca-b2c5-77500643d299'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.972691 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e27561d2dc240bbb0c2dde73e144349', 'request_id': 'req-274a3b9d-c463-44ca-b2c5-77500643d299'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.978257 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e27561d2dc240bbb0c2dde73e144349', 'request_id': 'req-274a3b9d-c463-44ca-b2c5-77500643d299'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.990723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] The token cache is being invalidate because project 7e27561d2dc240bbb0c2dde73e144349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:21.991013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.991631 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-274a3b9d-c463-44ca-b2c5-77500643d299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:21.993152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1304/2592] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:21 2022] DELETE /identity/v3/projects/7e27561d2dc240bbb0c2dde73e144349 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:21.998763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cfc80-cf0a-47e6-bbac-88fd635ef255 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:21.999210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cfc80-cf0a-47e6-bbac-88fd635ef255 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:21.999324 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cfc80-cf0a-47e6-bbac-88fd635ef255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.006140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:22.020491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a75cfc80-cf0a-47e6-bbac-88fd635ef255 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:22.022059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a75cfc80-cf0a-47e6-bbac-88fd635ef255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:22.079200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1289/2593] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:22.087019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad476bc5-dde0-417d-a69d-014700ce3db1 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:22.087254 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad476bc5-dde0-417d-a69d-014700ce3db1 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:22.087533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad476bc5-dde0-417d-a69d-014700ce3db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.105283 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-ad476bc5-dde0-417d-a69d-014700ce3db1 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:22.106516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ad476bc5-dde0-417d-a69d-014700ce3db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:22.115492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1305/2594] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:22.119762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cfc80-cf0a-47e6-bbac-88fd635ef255 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:22.130776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cfc80-cf0a-47e6-bbac-88fd635ef255 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:22.135315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb166d-b00c-4856-bb85-d7c23b8e25a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:22.135966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb166d-b00c-4856-bb85-d7c23b8e25a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:22.136249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb166d-b00c-4856-bb85-d7c23b8e25a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:22.136528 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb166d-b00c-4856-bb85-d7c23b8e25a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.137246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb166d-b00c-4856-bb85-d7c23b8e25a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:22.137555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb166d-b00c-4856-bb85-d7c23b8e25a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:22.139199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb166d-b00c-4856-bb85-d7c23b8e25a7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:22.150625 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1290/2595] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:22.154792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad476bc5-dde0-417d-a69d-014700ce3db1 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:22.163895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad476bc5-dde0-417d-a69d-014700ce3db1 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:22.168508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c0f9d-a48e-4115-91ce-0a473ecb7e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:22.169152 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c0f9d-a48e-4115-91ce-0a473ecb7e1f admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:22.169433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c0f9d-a48e-4115-91ce-0a473ecb7e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:22.169707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c0f9d-a48e-4115-91ce-0a473ecb7e1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.170389 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c0f9d-a48e-4115-91ce-0a473ecb7e1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:22.172332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c0f9d-a48e-4115-91ce-0a473ecb7e1f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:22.197645 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1306/2596] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:22 2022] POST /identity/v3/projects => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:22.201677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb166d-b00c-4856-bb85-d7c23b8e25a7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:22.211655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb166d-b00c-4856-bb85-d7c23b8e25a7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:22.266636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887bcf9-f515-4200-8801-1e223f5ebc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:22.267288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887bcf9-f515-4200-8801-1e223f5ebc98 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:22.267570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887bcf9-f515-4200-8801-1e223f5ebc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:22.267850 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887bcf9-f515-4200-8801-1e223f5ebc98 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.268533 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887bcf9-f515-4200-8801-1e223f5ebc98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:22.270604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3887bcf9-f515-4200-8801-1e223f5ebc98 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:22.289266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1291/2597] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:22 2022] POST /identity/v3/users => generated 284 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:22.293280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c0f9d-a48e-4115-91ce-0a473ecb7e1f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:22.303810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c0f9d-a48e-4115-91ce-0a473ecb7e1f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:22.308350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8ace1-2e25-456c-a5f8-b7560adf9b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:22.309062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8ace1-2e25-456c-a5f8-b7560adf9b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:22.309346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8ace1-2e25-456c-a5f8-b7560adf9b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:22.309624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8ace1-2e25-456c-a5f8-b7560adf9b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.310337 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8ace1-2e25-456c-a5f8-b7560adf9b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:22.311951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8ace1-2e25-456c-a5f8-b7560adf9b03 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:22.322744 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1307/2598] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:22.326383 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887bcf9-f515-4200-8801-1e223f5ebc98 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:22.336124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3887bcf9-f515-4200-8801-1e223f5ebc98 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:22.340692 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f922e-baa4-4e9c-9582-8a68b4a78037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:22.341270 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f922e-baa4-4e9c-9582-8a68b4a78037 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:22.341551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f922e-baa4-4e9c-9582-8a68b4a78037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:22.341827 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f922e-baa4-4e9c-9582-8a68b4a78037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.342578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f922e-baa4-4e9c-9582-8a68b4a78037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:22.344169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f922e-baa4-4e9c-9582-8a68b4a78037 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:22.353730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1292/2599] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:22.357569 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8ace1-2e25-456c-a5f8-b7560adf9b03 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:22.367991 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8ace1-2e25-456c-a5f8-b7560adf9b03 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:22.373824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd2b91-076a-48cd-85cf-9721286519c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:22.375280 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd2b91-076a-48cd-85cf-9721286519c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:22.375567 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd2b91-076a-48cd-85cf-9721286519c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:22.375845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd2b91-076a-48cd-85cf-9721286519c9 admin admin] PATH_INFO: `/v3/projects/66e8a58b94214f3ca8bb11f97a39785c/users/c7a031ee64f446d081c7403c329ea7e7/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.391578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd2b91-076a-48cd-85cf-9721286519c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66e8a58b94214f3ca8bb11f97a39785c, user_id=c7a031ee64f446d081c7403c329ea7e7, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:22.394310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd2b91-076a-48cd-85cf-9721286519c9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:22.407074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1308/2600] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:22 2022] PUT /identity/v3/projects/66e8a58b94214f3ca8bb11f97a39785c/users/c7a031ee64f446d081c7403c329ea7e7/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:22.417681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f702d46-5988-40b1-8b09-784307e821f1 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:22.417983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f702d46-5988-40b1-8b09-784307e821f1 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:22.418267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f702d46-5988-40b1-8b09-784307e821f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.444740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1f702d46-5988-40b1-8b09-784307e821f1 None None] MFA Rules not processed for user `c7a031ee64f446d081c7403c329ea7e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:22.445936 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1f702d46-5988-40b1-8b09-784307e821f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:22.503924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:22.512333 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1293/2601] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:22.521117 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd2b91-076a-48cd-85cf-9721286519c9 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:22.522835 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2acd2b91-076a-48cd-85cf-9721286519c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:22.541717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:22.581586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd2b91-076a-48cd-85cf-9721286519c9 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:22.585282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9d938-ee25-45e2-8819-1be167de985f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2c3bd8dd462846bf9423f8528aa5c489', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:22.587375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9d938-ee25-45e2-8819-1be167de985f service glance] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:22.587375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9d938-ee25-45e2-8819-1be167de985f service glance] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:22.587375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9d938-ee25-45e2-8819-1be167de985f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.596212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9d938-ee25-45e2-8819-1be167de985f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:22.598135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f9d938-ee25-45e2-8819-1be167de985f service glance] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:22.607074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1309/2602] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:32.510340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:32.603308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=19 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:34.331148 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f702d46-5988-40b1-8b09-784307e821f1 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:34.333036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1f702d46-5988-40b1-8b09-784307e821f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:34.390925 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f702d46-5988-40b1-8b09-784307e821f1 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:34.394768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdad295-1e95-429e-92e0-5272bd7471fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:34.396234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdad295-1e95-429e-92e0-5272bd7471fa service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:34.396523 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdad295-1e95-429e-92e0-5272bd7471fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:34.396895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdad295-1e95-429e-92e0-5272bd7471fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:34.398946 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1fdad295-1e95-429e-92e0-5272bd7471fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:34.464296 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdad295-1e95-429e-92e0-5272bd7471fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:34.465762 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdad295-1e95-429e-92e0-5272bd7471fa service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:34.476396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1294/2603] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:37.760012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9d938-ee25-45e2-8819-1be167de985f service glance] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:37.762069 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-11f9d938-ee25-45e2-8819-1be167de985f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:37.780665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f9d938-ee25-45e2-8819-1be167de985f service glance] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:37.785173 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:37.786626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:37.786912 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:37.787184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] PATH_INFO: `/v3/users/8145168b1d02438886cd0865844f7480` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:37.788361 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8145168b1d02438886cd0865844f7480)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:37.790270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:37.837654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8145168b1d02438886cd0865844f7480'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:37.849742 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8145168b1d02438886cd0865844f7480'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:37.857114 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8145168b1d02438886cd0865844f7480'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:37.857727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8145168b1d02438886cd0865844f7480'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:37.864863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-1fcaeec3-737b-4d50-aaab-91965fe43912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8145168b1d02438886cd0865844f7480'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:37.870766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1310/2604] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:37 2022] DELETE /identity/v3/users/8145168b1d02438886cd0865844f7480 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:38.039214 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdad295-1e95-429e-92e0-5272bd7471fa service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:38.052263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdad295-1e95-429e-92e0-5272bd7471fa service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:38.124445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:38.125599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.125884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.126164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] PATH_INFO: `/v3/projects/ba383df93dcf4c32b2b1c17bebd7f752` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.127932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba383df93dcf4c32b2b1c17bebd7f752)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:38.131204 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:38.183549 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba383df93dcf4c32b2b1c17bebd7f752', 'request_id': 'req-6782aba9-fe15-4259-a2b5-49f518e6a449'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:38.185559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:38.190104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba383df93dcf4c32b2b1c17bebd7f752', 'request_id': 'req-6782aba9-fe15-4259-a2b5-49f518e6a449'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:38.197263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba383df93dcf4c32b2b1c17bebd7f752', 'request_id': 'req-6782aba9-fe15-4259-a2b5-49f518e6a449'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:38.216158 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] The token cache is being invalidate because project ba383df93dcf4c32b2b1c17bebd7f752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:38.216654 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:38.217286 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6782aba9-fe15-4259-a2b5-49f518e6a449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:38.219162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1295/2605] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:38 2022] DELETE /identity/v3/projects/ba383df93dcf4c32b2b1c17bebd7f752 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:38.224094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b932fc-85bb-4ded-b35a-dbbc35b1e425 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.224412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b932fc-85bb-4ded-b35a-dbbc35b1e425 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.224832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b932fc-85bb-4ded-b35a-dbbc35b1e425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.243137 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-25b932fc-85bb-4ded-b35a-dbbc35b1e425 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:38.244407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-25b932fc-85bb-4ded-b35a-dbbc35b1e425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:38.304972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1311/2606] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:11:38.313079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98797c62-c88c-4541-a8aa-3930de24115c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.313373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98797c62-c88c-4541-a8aa-3930de24115c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.313666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98797c62-c88c-4541-a8aa-3930de24115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.337181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-98797c62-c88c-4541-a8aa-3930de24115c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:38.338819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-98797c62-c88c-4541-a8aa-3930de24115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:38.353381 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1296/2607] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:38.354716 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b932fc-85bb-4ded-b35a-dbbc35b1e425 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:38.365355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b932fc-85bb-4ded-b35a-dbbc35b1e425 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:38.369990 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f781e-61ca-4f51-8ae7-9cda28408fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:38.370846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f781e-61ca-4f51-8ae7-9cda28408fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.371135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f781e-61ca-4f51-8ae7-9cda28408fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.371418 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f781e-61ca-4f51-8ae7-9cda28408fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.372154 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f781e-61ca-4f51-8ae7-9cda28408fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:38.372464 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f781e-61ca-4f51-8ae7-9cda28408fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:38.374150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f781e-61ca-4f51-8ae7-9cda28408fe7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:38.386482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1312/2608] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:38.390787 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98797c62-c88c-4541-a8aa-3930de24115c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:38.400656 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98797c62-c88c-4541-a8aa-3930de24115c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:38.405249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adbc1b-4239-4c7a-8c23-6b2f5a3b2309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:38.405896 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adbc1b-4239-4c7a-8c23-6b2f5a3b2309 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.406175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adbc1b-4239-4c7a-8c23-6b2f5a3b2309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.406465 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adbc1b-4239-4c7a-8c23-6b2f5a3b2309 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.407175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adbc1b-4239-4c7a-8c23-6b2f5a3b2309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:38.409070 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adbc1b-4239-4c7a-8c23-6b2f5a3b2309 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:38.439802 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1297/2609] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:38 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:38.441277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f781e-61ca-4f51-8ae7-9cda28408fe7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:38.451874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f781e-61ca-4f51-8ae7-9cda28408fe7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:38.513638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988d8cd-a5e7-43f8-befd-3550698af35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:38.513638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988d8cd-a5e7-43f8-befd-3550698af35a admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.513638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988d8cd-a5e7-43f8-befd-3550698af35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.513638 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988d8cd-a5e7-43f8-befd-3550698af35a admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.514053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988d8cd-a5e7-43f8-befd-3550698af35a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:38.514255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988d8cd-a5e7-43f8-befd-3550698af35a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:38.521009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:38.535609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1313/2610] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:38 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:38.539617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adbc1b-4239-4c7a-8c23-6b2f5a3b2309 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:38.551359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adbc1b-4239-4c7a-8c23-6b2f5a3b2309 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:38.555909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cb7a4-1b9f-4b51-bcef-62b66946f5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:38.556666 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cb7a4-1b9f-4b51-bcef-62b66946f5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.556826 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cb7a4-1b9f-4b51-bcef-62b66946f5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.557098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9cb7a4-1b9f-4b51-bcef-62b66946f5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.557949 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9cb7a4-1b9f-4b51-bcef-62b66946f5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:38.559577 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9cb7a4-1b9f-4b51-bcef-62b66946f5b8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:38.570605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1298/2611] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:38.574862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988d8cd-a5e7-43f8-befd-3550698af35a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:38.585094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988d8cd-a5e7-43f8-befd-3550698af35a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:38.589790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe12075-eae5-499e-af73-8b89cc7aab3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:38.590479 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe12075-eae5-499e-af73-8b89cc7aab3f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.590775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe12075-eae5-499e-af73-8b89cc7aab3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.591058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe12075-eae5-499e-af73-8b89cc7aab3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.591788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe12075-eae5-499e-af73-8b89cc7aab3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:38.593387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe12075-eae5-499e-af73-8b89cc7aab3f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:38.603791 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1314/2612] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:38.607941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cb7a4-1b9f-4b51-bcef-62b66946f5b8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:38.618773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9cb7a4-1b9f-4b51-bcef-62b66946f5b8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:38.623329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997a3b6-fa23-4c6b-89ea-9f6ea29a0337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:38.624193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997a3b6-fa23-4c6b-89ea-9f6ea29a0337 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.624470 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997a3b6-fa23-4c6b-89ea-9f6ea29a0337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.624742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997a3b6-fa23-4c6b-89ea-9f6ea29a0337 admin admin] PATH_INFO: `/v3/projects/1ae411e018ff408ca5a993430a1b3e7a/users/7a10884f68894e14b36c5f8849812a42/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.640273 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997a3b6-fa23-4c6b-89ea-9f6ea29a0337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae411e018ff408ca5a993430a1b3e7a, user_id=7a10884f68894e14b36c5f8849812a42, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:38.642958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997a3b6-fa23-4c6b-89ea-9f6ea29a0337 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:38.655423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1299/2613] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:38 2022] PUT /identity/v3/projects/1ae411e018ff408ca5a993430a1b3e7a/users/7a10884f68894e14b36c5f8849812a42/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:38.666128 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d3b6c-60bc-4061-9d66-17ae009e749e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.666424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d3b6c-60bc-4061-9d66-17ae009e749e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.666740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d3b6c-60bc-4061-9d66-17ae009e749e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.696374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-2f6d3b6c-60bc-4061-9d66-17ae009e749e None None] MFA Rules not processed for user `7a10884f68894e14b36c5f8849812a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:38.697617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2f6d3b6c-60bc-4061-9d66-17ae009e749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:38.760043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1315/2614] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:38.770582 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997a3b6-fa23-4c6b-89ea-9f6ea29a0337 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:38.772267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b997a3b6-fa23-4c6b-89ea-9f6ea29a0337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:38.851085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997a3b6-fa23-4c6b-89ea-9f6ea29a0337 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:38.856741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a1d74-5148-4620-9954-59bdffa9868d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:38.857331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a1d74-5148-4620-9954-59bdffa9868d service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.857622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a1d74-5148-4620-9954-59bdffa9868d service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.857903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a1d74-5148-4620-9954-59bdffa9868d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.866435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a1d74-5148-4620-9954-59bdffa9868d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:38.868075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a1d74-5148-4620-9954-59bdffa9868d service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:38.876895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1300/2615] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:41.533500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d3b6c-60bc-4061-9d66-17ae009e749e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:41.598754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d3b6c-60bc-4061-9d66-17ae009e749e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:41.603321 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:41.604229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:41.604509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:41.604789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] PATH_INFO: `/v3/users/7a10884f68894e14b36c5f8849812a42` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:41.605976 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a10884f68894e14b36c5f8849812a42)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:41.607920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:41.660719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a10884f68894e14b36c5f8849812a42'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:41.676223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a10884f68894e14b36c5f8849812a42'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:41.680306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:41.685235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a10884f68894e14b36c5f8849812a42'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:41.685987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a10884f68894e14b36c5f8849812a42'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:41.694698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a10884f68894e14b36c5f8849812a42'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:41.701170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1316/2616] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:41 2022] DELETE /identity/v3/users/7a10884f68894e14b36c5f8849812a42 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:41.707194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eea0a-e8ee-4c59-912e-05e896242f30 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:41.707677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eea0a-e8ee-4c59-912e-05e896242f30 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:41.707791 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eea0a-e8ee-4c59-912e-05e896242f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:41.725075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d96eea0a-e8ee-4c59-912e-05e896242f30 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:41.726323 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d96eea0a-e8ee-4c59-912e-05e896242f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:41.785940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1301/2617] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:41.795398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:41.797074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:41.860437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e2812-46a8-4c5e-b812-bbee0330b0a5 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:41.864968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67dd36-1308-4b47-9c89-ba449052d762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:41.865621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67dd36-1308-4b47-9c89-ba449052d762 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:41.865900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67dd36-1308-4b47-9c89-ba449052d762 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:41.866169 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67dd36-1308-4b47-9c89-ba449052d762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:41.875730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67dd36-1308-4b47-9c89-ba449052d762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:41.877162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67dd36-1308-4b47-9c89-ba449052d762 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:41.887747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1317/2618] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:42.237116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96eea0a-e8ee-4c59-912e-05e896242f30 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:42.239730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d96eea0a-e8ee-4c59-912e-05e896242f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:42.265819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:42.288365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96eea0a-e8ee-4c59-912e-05e896242f30 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:42.296670 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:42.298294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.299065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.299811 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] PATH_INFO: `/v3/projects/1ae411e018ff408ca5a993430a1b3e7a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.301469 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae411e018ff408ca5a993430a1b3e7a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:42.303778 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:42.361495 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae411e018ff408ca5a993430a1b3e7a', 'request_id': 'req-a2f9f2dd-fc71-4c00-952b-a96513f8299c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:42.367398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae411e018ff408ca5a993430a1b3e7a', 'request_id': 'req-a2f9f2dd-fc71-4c00-952b-a96513f8299c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:42.373623 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae411e018ff408ca5a993430a1b3e7a', 'request_id': 'req-a2f9f2dd-fc71-4c00-952b-a96513f8299c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:42.385439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] The token cache is being invalidate because project 1ae411e018ff408ca5a993430a1b3e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:42.385730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:42.386349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a2f9f2dd-fc71-4c00-952b-a96513f8299c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:42.387939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1302/2619] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:42 2022] DELETE /identity/v3/projects/1ae411e018ff408ca5a993430a1b3e7a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:42.392988 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e93c6-7492-4f70-9658-19ec27d62b58 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.393276 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e93c6-7492-4f70-9658-19ec27d62b58 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.393551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e93c6-7492-4f70-9658-19ec27d62b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.412023 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-709e93c6-7492-4f70-9658-19ec27d62b58 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:42.413210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-709e93c6-7492-4f70-9658-19ec27d62b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:42.470292 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1318/2620] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:42.478630 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdcf4e-8d97-4aaa-a0ee-ea3114a37e5c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.478926 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdcf4e-8d97-4aaa-a0ee-ea3114a37e5c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.479208 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdcf4e-8d97-4aaa-a0ee-ea3114a37e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.497744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6dbdcf4e-8d97-4aaa-a0ee-ea3114a37e5c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:42.499007 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6dbdcf4e-8d97-4aaa-a0ee-ea3114a37e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:42.506948 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1303/2621] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:11:42.511127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e93c6-7492-4f70-9658-19ec27d62b58 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:42.521171 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e93c6-7492-4f70-9658-19ec27d62b58 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:42.525746 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8f90a-2df0-4fab-bac4-55d98dd62e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:42.526430 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8f90a-2df0-4fab-bac4-55d98dd62e03 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.526745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8f90a-2df0-4fab-bac4-55d98dd62e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.527026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8f90a-2df0-4fab-bac4-55d98dd62e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.527758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8f90a-2df0-4fab-bac4-55d98dd62e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:42.528068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8f90a-2df0-4fab-bac4-55d98dd62e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:42.529699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8f90a-2df0-4fab-bac4-55d98dd62e03 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:42.541816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1319/2622] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:42.546069 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdcf4e-8d97-4aaa-a0ee-ea3114a37e5c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:42.555319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdcf4e-8d97-4aaa-a0ee-ea3114a37e5c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:42.559900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612a11d-8f28-45da-844e-ed1a014f0407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:42.560539 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612a11d-8f28-45da-844e-ed1a014f0407 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.560821 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612a11d-8f28-45da-844e-ed1a014f0407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.561096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612a11d-8f28-45da-844e-ed1a014f0407 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.561777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612a11d-8f28-45da-844e-ed1a014f0407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:42.563723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612a11d-8f28-45da-844e-ed1a014f0407 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:42.588838 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1304/2623] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:42 2022] POST /identity/v3/projects => generated 405 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:42.593126 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8f90a-2df0-4fab-bac4-55d98dd62e03 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:42.604041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8f90a-2df0-4fab-bac4-55d98dd62e03 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:42.663753 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a69b7-dced-4a37-96aa-2dfae35152ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:42.664465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a69b7-dced-4a37-96aa-2dfae35152ef admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.664750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a69b7-dced-4a37-96aa-2dfae35152ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.665026 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a69b7-dced-4a37-96aa-2dfae35152ef admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.665739 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a69b7-dced-4a37-96aa-2dfae35152ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:42.667677 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493a69b7-dced-4a37-96aa-2dfae35152ef admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:42.691150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1320/2624] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:42 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:42.695793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612a11d-8f28-45da-844e-ed1a014f0407 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:42.706075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612a11d-8f28-45da-844e-ed1a014f0407 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:42.711396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac550d29-dc1e-421a-830d-6ca40e930c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:42.712046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac550d29-dc1e-421a-830d-6ca40e930c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.712331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac550d29-dc1e-421a-830d-6ca40e930c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.712609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac550d29-dc1e-421a-830d-6ca40e930c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.713334 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac550d29-dc1e-421a-830d-6ca40e930c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:42.714985 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac550d29-dc1e-421a-830d-6ca40e930c4d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:42.725196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1305/2625] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:42.729174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a69b7-dced-4a37-96aa-2dfae35152ef admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:42.739105 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a69b7-dced-4a37-96aa-2dfae35152ef admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:42.744379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a24a8-0c55-4309-970b-f0aac672f4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:42.745030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a24a8-0c55-4309-970b-f0aac672f4ba admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.745315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a24a8-0c55-4309-970b-f0aac672f4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.745589 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a24a8-0c55-4309-970b-f0aac672f4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.746334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a24a8-0c55-4309-970b-f0aac672f4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:42.747961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a24a8-0c55-4309-970b-f0aac672f4ba admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:42.749220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:42.760335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1321/2626] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:42.764346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac550d29-dc1e-421a-830d-6ca40e930c4d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:42.775246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac550d29-dc1e-421a-830d-6ca40e930c4d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:42.780076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54620f-d94c-4615-9c4c-c698120a3288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:42.780963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54620f-d94c-4615-9c4c-c698120a3288 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.781247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54620f-d94c-4615-9c4c-c698120a3288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.781524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a54620f-d94c-4615-9c4c-c698120a3288 admin admin] PATH_INFO: `/v3/projects/354e149c023f414f88f28f348dd47889/users/f2f8250c55a24fc78ba807226914ada6/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.799690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54620f-d94c-4615-9c4c-c698120a3288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=354e149c023f414f88f28f348dd47889, user_id=f2f8250c55a24fc78ba807226914ada6, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:42.803727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a54620f-d94c-4615-9c4c-c698120a3288 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:42.825421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1306/2627] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:42 2022] PUT /identity/v3/projects/354e149c023f414f88f28f348dd47889/users/f2f8250c55a24fc78ba807226914ada6/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:42.838379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6f29a-fd5f-4fb7-9c08-8295e450fecd None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.838975 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6f29a-fd5f-4fb7-9c08-8295e450fecd None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.839256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6f29a-fd5f-4fb7-9c08-8295e450fecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.869057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-40d6f29a-fd5f-4fb7-9c08-8295e450fecd None None] MFA Rules not processed for user `f2f8250c55a24fc78ba807226914ada6`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:42.870265 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-40d6f29a-fd5f-4fb7-9c08-8295e450fecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:42.942219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1322/2628] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:42 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:42.943805 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54620f-d94c-4615-9c4c-c698120a3288 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:43.034823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a54620f-d94c-4615-9c4c-c698120a3288 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:43.041250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5778ca-5a96-4e80-838d-b665df8a026f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:43.042053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5778ca-5a96-4e80-838d-b665df8a026f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:43.042439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5778ca-5a96-4e80-838d-b665df8a026f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:43.042844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5778ca-5a96-4e80-838d-b665df8a026f admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:43.043635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5778ca-5a96-4e80-838d-b665df8a026f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:43.045663 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5778ca-5a96-4e80-838d-b665df8a026f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:43.078885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1307/2629] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:42 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:43.082930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6f29a-fd5f-4fb7-9c08-8295e450fecd None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:43.095348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6f29a-fd5f-4fb7-9c08-8295e450fecd None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:43.163841 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06a841-92d1-43b0-a107-1579242146ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:43.164724 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06a841-92d1-43b0-a107-1579242146ec admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:43.165156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06a841-92d1-43b0-a107-1579242146ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:43.165590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06a841-92d1-43b0-a107-1579242146ec admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:43.166471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06a841-92d1-43b0-a107-1579242146ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:43.168533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06a841-92d1-43b0-a107-1579242146ec admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:43.192207 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1323/2630] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:43 2022] POST /identity/v3/users => generated 309 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:11:43.196176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5778ca-5a96-4e80-838d-b665df8a026f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:43.207268 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5778ca-5a96-4e80-838d-b665df8a026f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:43.212121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63547189-a59b-4e57-bc85-ba7d9a95816a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:43.212793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63547189-a59b-4e57-bc85-ba7d9a95816a admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:43.213077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63547189-a59b-4e57-bc85-ba7d9a95816a admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:43.213390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63547189-a59b-4e57-bc85-ba7d9a95816a admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:43.214112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63547189-a59b-4e57-bc85-ba7d9a95816a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:43.215801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63547189-a59b-4e57-bc85-ba7d9a95816a admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:43.217565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:43.230266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1308/2631] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:43.236135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06a841-92d1-43b0-a107-1579242146ec admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:43.252334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06a841-92d1-43b0-a107-1579242146ec admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:43.259772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa90f2-6d9f-444f-bb16-e0b9d3f9c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:43.260667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa90f2-6d9f-444f-bb16-e0b9d3f9c927 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:43.260981 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa90f2-6d9f-444f-bb16-e0b9d3f9c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:43.261261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa90f2-6d9f-444f-bb16-e0b9d3f9c927 admin admin] PATH_INFO: `/v3/projects/a9353eac374b4b0d9f3eecedfc13db4a/users/a8241b0f49874b219dd5c86f1c39280e/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:43.277274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa90f2-6d9f-444f-bb16-e0b9d3f9c927 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9353eac374b4b0d9f3eecedfc13db4a, user_id=a8241b0f49874b219dd5c86f1c39280e, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:43.279974 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa90f2-6d9f-444f-bb16-e0b9d3f9c927 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:43.293099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1324/2632] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:43 2022] PUT /identity/v3/projects/a9353eac374b4b0d9f3eecedfc13db4a/users/a8241b0f49874b219dd5c86f1c39280e/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:43.297078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63547189-a59b-4e57-bc85-ba7d9a95816a admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:43.308048 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63547189-a59b-4e57-bc85-ba7d9a95816a admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:43.384318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac3ca1-2c7f-4555-9299-89b446bb8f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:43.385122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac3ca1-2c7f-4555-9299-89b446bb8f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:43.385478 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac3ca1-2c7f-4555-9299-89b446bb8f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:43.385923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac3ca1-2c7f-4555-9299-89b446bb8f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:43.386829 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac3ca1-2c7f-4555-9299-89b446bb8f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:43.388751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac3ca1-2c7f-4555-9299-89b446bb8f1f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:43.405357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1309/2633] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:43.417012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa90f2-6d9f-444f-bb16-e0b9d3f9c927 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:43.421102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa90f2-6d9f-444f-bb16-e0b9d3f9c927 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:43.433121 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38cd0a-ef7f-45a4-ba61-f8cb4cd04483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:43.434229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38cd0a-ef7f-45a4-ba61-f8cb4cd04483 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:43.434483 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38cd0a-ef7f-45a4-ba61-f8cb4cd04483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:43.434704 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38cd0a-ef7f-45a4-ba61-f8cb4cd04483 admin admin] PATH_INFO: `/v3/projects/a9353eac374b4b0d9f3eecedfc13db4a/users/a8241b0f49874b219dd5c86f1c39280e/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:43.436763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38cd0a-ef7f-45a4-ba61-f8cb4cd04483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9353eac374b4b0d9f3eecedfc13db4a, user_id=a8241b0f49874b219dd5c86f1c39280e, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:43.439437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38cd0a-ef7f-45a4-ba61-f8cb4cd04483 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:43.450266 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1325/2634] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:43 2022] PUT /identity/v3/projects/a9353eac374b4b0d9f3eecedfc13db4a/users/a8241b0f49874b219dd5c86f1c39280e/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:43.460482 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24a5c2-1207-4e18-b9d9-59698c928bad None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:43.460873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24a5c2-1207-4e18-b9d9-59698c928bad None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:43.461266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24a5c2-1207-4e18-b9d9-59698c928bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:43.489912 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9d24a5c2-1207-4e18-b9d9-59698c928bad None None] MFA Rules not processed for user `a8241b0f49874b219dd5c86f1c39280e`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:43.491166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9d24a5c2-1207-4e18-b9d9-59698c928bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:43.561490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1310/2635] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:43.576155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38cd0a-ef7f-45a4-ba61-f8cb4cd04483 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:43.577429 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5e38cd0a-ef7f-45a4-ba61-f8cb4cd04483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:43.635514 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:43.653131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38cd0a-ef7f-45a4-ba61-f8cb4cd04483 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:43.657227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53782a-0e4c-40ba-a8b7-73a2abee4c1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:43.658013 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53782a-0e4c-40ba-a8b7-73a2abee4c1e service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:43.658463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53782a-0e4c-40ba-a8b7-73a2abee4c1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:43.658870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53782a-0e4c-40ba-a8b7-73a2abee4c1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:43.669712 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53782a-0e4c-40ba-a8b7-73a2abee4c1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:43.671255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53782a-0e4c-40ba-a8b7-73a2abee4c1e service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:43.682245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1326/2636] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:43 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:45.480073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24a5c2-1207-4e18-b9d9-59698c928bad None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:45.557432 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24a5c2-1207-4e18-b9d9-59698c928bad None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:45.562909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:45.563773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:45.564089 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:45.564447 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] PATH_INFO: `/v3/users/f2f8250c55a24fc78ba807226914ada6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:45.565870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2f8250c55a24fc78ba807226914ada6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:45.568686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:45.590947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:45.621864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2f8250c55a24fc78ba807226914ada6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:45.622557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2f8250c55a24fc78ba807226914ada6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:45.631261 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2f8250c55a24fc78ba807226914ada6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:45.631891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2f8250c55a24fc78ba807226914ada6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:45.644271 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2f8250c55a24fc78ba807226914ada6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:45.657213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1311/2637] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:45 2022] DELETE /identity/v3/users/f2f8250c55a24fc78ba807226914ada6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:45.661747 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3322be-7adc-4967-8f12-fec2a524391f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:45.662215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3322be-7adc-4967-8f12-fec2a524391f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:45.662316 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3322be-7adc-4967-8f12-fec2a524391f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:45.680040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-9d3322be-7adc-4967-8f12-fec2a524391f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:45.681247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9d3322be-7adc-4967-8f12-fec2a524391f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:45.740706 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1327/2638] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:45.750516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:45.752021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:45.810746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9eb604-be68-483e-9c51-5da3d509f8de admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:45.814555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8bb1b-190a-467b-9491-cd37b2135ef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:45.815301 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8bb1b-190a-467b-9491-cd37b2135ef8 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:45.815653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8bb1b-190a-467b-9491-cd37b2135ef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:45.815998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8bb1b-190a-467b-9491-cd37b2135ef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:45.827632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8bb1b-190a-467b-9491-cd37b2135ef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:45.829140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8bb1b-190a-467b-9491-cd37b2135ef8 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:45.841137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1312/2639] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:46.102164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3322be-7adc-4967-8f12-fec2a524391f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:46.103667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9d3322be-7adc-4967-8f12-fec2a524391f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:46.117334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3322be-7adc-4967-8f12-fec2a524391f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:46.123062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:46.124818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:46.125297 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:46.125580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] PATH_INFO: `/v3/projects/354e149c023f414f88f28f348dd47889` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:46.126930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=354e149c023f414f88f28f348dd47889)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:46.129397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:46.174532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354e149c023f414f88f28f348dd47889', 'request_id': 'req-8e549d77-ac7d-4e39-b63a-926b1f051328'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.181079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354e149c023f414f88f28f348dd47889', 'request_id': 'req-8e549d77-ac7d-4e39-b63a-926b1f051328'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.186696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354e149c023f414f88f28f348dd47889', 'request_id': 'req-8e549d77-ac7d-4e39-b63a-926b1f051328'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.199525 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] The token cache is being invalidate because project 354e149c023f414f88f28f348dd47889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:46.199842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.200465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.202024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1328/2640] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:46 2022] DELETE /identity/v3/projects/354e149c023f414f88f28f348dd47889 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:46.206038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8bb1b-190a-467b-9491-cd37b2135ef8 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:46.215761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8bb1b-190a-467b-9491-cd37b2135ef8 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:46.272176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:46.273022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:46.273304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:46.273582 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] PATH_INFO: `/v3/users/a8241b0f49874b219dd5c86f1c39280e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:46.274845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8241b0f49874b219dd5c86f1c39280e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:46.276820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:46.318018 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:46.323343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8241b0f49874b219dd5c86f1c39280e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.324275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8241b0f49874b219dd5c86f1c39280e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.332719 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8241b0f49874b219dd5c86f1c39280e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.333414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8241b0f49874b219dd5c86f1c39280e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.345676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9eff7f7-89c6-44bc-b516-c6a471cc1db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8241b0f49874b219dd5c86f1c39280e'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.357551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1313/2641] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:46 2022] DELETE /identity/v3/users/a8241b0f49874b219dd5c86f1c39280e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:46.623502 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:46.625010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:46.638503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549d77-ac7d-4e39-b63a-926b1f051328 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:46.699858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:46.701227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:46.701591 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:46.701873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] PATH_INFO: `/v3/projects/a9353eac374b4b0d9f3eecedfc13db4a` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:46.703119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9353eac374b4b0d9f3eecedfc13db4a)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:46.705069 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:46.711817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:46.755259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9353eac374b4b0d9f3eecedfc13db4a', 'request_id': 'req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.761577 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9353eac374b4b0d9f3eecedfc13db4a', 'request_id': 'req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.767399 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9353eac374b4b0d9f3eecedfc13db4a', 'request_id': 'req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.781407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] The token cache is being invalidate because project a9353eac374b4b0d9f3eecedfc13db4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:46.781690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.782392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be7bbf4e-2d9d-4b38-93cc-d63956b2114d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:46.784095 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1329/2642] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:46 2022] DELETE /identity/v3/projects/a9353eac374b4b0d9f3eecedfc13db4a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:46.790585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec204fb5-27b4-4eb3-aca4-41190f3da21b None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:46.791037 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec204fb5-27b4-4eb3-aca4-41190f3da21b None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:46.791132 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec204fb5-27b4-4eb3-aca4-41190f3da21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:46.809650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ec204fb5-27b4-4eb3-aca4-41190f3da21b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:46.810883 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ec204fb5-27b4-4eb3-aca4-41190f3da21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:46.869904 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1314/2643] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:46.877643 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67561c2-e1d4-4fec-aa02-7b94e5732511 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:46.877938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67561c2-e1d4-4fec-aa02-7b94e5732511 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:46.878220 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67561c2-e1d4-4fec-aa02-7b94e5732511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:46.896613 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-a67561c2-e1d4-4fec-aa02-7b94e5732511 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:46.897804 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a67561c2-e1d4-4fec-aa02-7b94e5732511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:46.905603 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1330/2644] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:46.909557 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec204fb5-27b4-4eb3-aca4-41190f3da21b None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:46.919699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec204fb5-27b4-4eb3-aca4-41190f3da21b None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:46.924212 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f8df1-0235-4a5c-9647-6644bd8c6837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:46.924856 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8df1-0235-4a5c-9647-6644bd8c6837 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:46.925144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8df1-0235-4a5c-9647-6644bd8c6837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:46.925423 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f8df1-0235-4a5c-9647-6644bd8c6837 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:46.926178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f8df1-0235-4a5c-9647-6644bd8c6837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:46.926525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f8df1-0235-4a5c-9647-6644bd8c6837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:46.928136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f8df1-0235-4a5c-9647-6644bd8c6837 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:46.939329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1315/2645] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:46.943372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67561c2-e1d4-4fec-aa02-7b94e5732511 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:46.952554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67561c2-e1d4-4fec-aa02-7b94e5732511 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:46.957705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe62b61-ec02-41ac-bd5a-7f5a7b5ed142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:46.958351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe62b61-ec02-41ac-bd5a-7f5a7b5ed142 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:46.958673 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe62b61-ec02-41ac-bd5a-7f5a7b5ed142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:46.958951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe62b61-ec02-41ac-bd5a-7f5a7b5ed142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:46.959635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe62b61-ec02-41ac-bd5a-7f5a7b5ed142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:46.961520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe62b61-ec02-41ac-bd5a-7f5a7b5ed142 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:46.987179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1331/2646] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:46 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:46.991384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f8df1-0235-4a5c-9647-6644bd8c6837 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.001216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f8df1-0235-4a5c-9647-6644bd8c6837 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.055853 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786de3a4-637b-4ecb-ae14-74003b63610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.056512 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786de3a4-637b-4ecb-ae14-74003b63610f admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.056799 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786de3a4-637b-4ecb-ae14-74003b63610f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.057079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786de3a4-637b-4ecb-ae14-74003b63610f admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.057771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786de3a4-637b-4ecb-ae14-74003b63610f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.059727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786de3a4-637b-4ecb-ae14-74003b63610f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.078229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1316/2647] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:46 2022] POST /identity/v3/users => generated 284 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:47.082887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe62b61-ec02-41ac-bd5a-7f5a7b5ed142 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.093011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe62b61-ec02-41ac-bd5a-7f5a7b5ed142 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.098057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd2296-7900-4b22-9977-e5a4e025191e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.098767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd2296-7900-4b22-9977-e5a4e025191e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.099055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd2296-7900-4b22-9977-e5a4e025191e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.099325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd2296-7900-4b22-9977-e5a4e025191e admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.100055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd2296-7900-4b22-9977-e5a4e025191e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.101642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd2296-7900-4b22-9977-e5a4e025191e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.111817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1332/2648] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:47.115793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786de3a4-637b-4ecb-ae14-74003b63610f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.125593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786de3a4-637b-4ecb-ae14-74003b63610f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.131038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7b3b1-2ea2-46ac-b228-c7a6f5793b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.131683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7b3b1-2ea2-46ac-b228-c7a6f5793b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.131969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7b3b1-2ea2-46ac-b228-c7a6f5793b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.132250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7b3b1-2ea2-46ac-b228-c7a6f5793b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.132972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7b3b1-2ea2-46ac-b228-c7a6f5793b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.134599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7b3b1-2ea2-46ac-b228-c7a6f5793b45 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.145631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1317/2649] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:47.149573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd2296-7900-4b22-9977-e5a4e025191e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.159348 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd2296-7900-4b22-9977-e5a4e025191e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.164037 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b448f8-d694-476a-802e-3af39ec58c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.164903 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b448f8-d694-476a-802e-3af39ec58c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.165191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b448f8-d694-476a-802e-3af39ec58c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.165467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b448f8-d694-476a-802e-3af39ec58c19 admin admin] PATH_INFO: `/v3/projects/fbced3dddcc545e4aee8c98e6c8b0a45/users/6510447467514897bd9c4802d38d4a22/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.179877 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b448f8-d694-476a-802e-3af39ec58c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbced3dddcc545e4aee8c98e6c8b0a45, user_id=6510447467514897bd9c4802d38d4a22, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.182559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b448f8-d694-476a-802e-3af39ec58c19 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.195848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1333/2650] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:47 2022] PUT /identity/v3/projects/fbced3dddcc545e4aee8c98e6c8b0a45/users/6510447467514897bd9c4802d38d4a22/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:11:47.207675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31d336-fc2c-4c43-8ddc-d1f4d31c5249 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.207969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31d336-fc2c-4c43-8ddc-d1f4d31c5249 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.208251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31d336-fc2c-4c43-8ddc-d1f4d31c5249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.233542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dd31d336-fc2c-4c43-8ddc-d1f4d31c5249 None None] MFA Rules not processed for user `6510447467514897bd9c4802d38d4a22`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:47.234763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dd31d336-fc2c-4c43-8ddc-d1f4d31c5249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:47.246595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:47.301703 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1318/2651] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:47.306267 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b448f8-d694-476a-802e-3af39ec58c19 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.366980 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b448f8-d694-476a-802e-3af39ec58c19 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.371565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192cbc7-973d-4088-a66a-5992eb2b8ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.372223 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192cbc7-973d-4088-a66a-5992eb2b8ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.372530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192cbc7-973d-4088-a66a-5992eb2b8ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.372786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192cbc7-973d-4088-a66a-5992eb2b8ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.373474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192cbc7-973d-4088-a66a-5992eb2b8ef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.375411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192cbc7-973d-4088-a66a-5992eb2b8ef6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.400950 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1334/2652] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:47 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:47.405027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31d336-fc2c-4c43-8ddc-d1f4d31c5249 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.415702 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31d336-fc2c-4c43-8ddc-d1f4d31c5249 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.475279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e701fe-610e-48bc-a241-d136c869cb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.475950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e701fe-610e-48bc-a241-d136c869cb11 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.476283 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e701fe-610e-48bc-a241-d136c869cb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.476576 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e701fe-610e-48bc-a241-d136c869cb11 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.477331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e701fe-610e-48bc-a241-d136c869cb11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.479392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e701fe-610e-48bc-a241-d136c869cb11 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.500498 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1319/2653] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:47 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:47.505118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192cbc7-973d-4088-a66a-5992eb2b8ef6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.515869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192cbc7-973d-4088-a66a-5992eb2b8ef6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.520554 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2783588-7677-49f5-b9bd-9482400eba45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.521213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2783588-7677-49f5-b9bd-9482400eba45 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.521501 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2783588-7677-49f5-b9bd-9482400eba45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.521782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2783588-7677-49f5-b9bd-9482400eba45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.522538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2783588-7677-49f5-b9bd-9482400eba45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.524135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2783588-7677-49f5-b9bd-9482400eba45 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.534618 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1335/2654] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:47.538790 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e701fe-610e-48bc-a241-d136c869cb11 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.548784 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e701fe-610e-48bc-a241-d136c869cb11 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.553916 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9e485-2841-49f4-a7b1-a6a18d6c6eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.554822 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9e485-2841-49f4-a7b1-a6a18d6c6eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.555108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9e485-2841-49f4-a7b1-a6a18d6c6eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.555390 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd9e485-2841-49f4-a7b1-a6a18d6c6eb7 admin admin] PATH_INFO: `/v3/projects/b1179882152342bcbbd62aca69dccfcc/users/6ea04e1ba0584f04a1e19c46821f3eef/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.570636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9e485-2841-49f4-a7b1-a6a18d6c6eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1179882152342bcbbd62aca69dccfcc, user_id=6ea04e1ba0584f04a1e19c46821f3eef, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.573245 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd9e485-2841-49f4-a7b1-a6a18d6c6eb7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.585764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1320/2655] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:47 2022] PUT /identity/v3/projects/b1179882152342bcbbd62aca69dccfcc/users/6ea04e1ba0584f04a1e19c46821f3eef/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:11:47.589511 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2783588-7677-49f5-b9bd-9482400eba45 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.596701 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:47.604455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2783588-7677-49f5-b9bd-9482400eba45 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.664092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34585aa4-fb7f-4303-a876-10124165a87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.664755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34585aa4-fb7f-4303-a876-10124165a87f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.665038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34585aa4-fb7f-4303-a876-10124165a87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.665314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34585aa4-fb7f-4303-a876-10124165a87f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.666021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34585aa4-fb7f-4303-a876-10124165a87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.667636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34585aa4-fb7f-4303-a876-10124165a87f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.677557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1336/2656] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:47.682184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9e485-2841-49f4-a7b1-a6a18d6c6eb7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.693140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd9e485-2841-49f4-a7b1-a6a18d6c6eb7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.697724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904e418-31ca-4d0b-a3b7-8a4ef439e575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.698622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904e418-31ca-4d0b-a3b7-8a4ef439e575 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.698905 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904e418-31ca-4d0b-a3b7-8a4ef439e575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.699180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3904e418-31ca-4d0b-a3b7-8a4ef439e575 admin admin] PATH_INFO: `/v3/projects/b1179882152342bcbbd62aca69dccfcc/users/6ea04e1ba0584f04a1e19c46821f3eef/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.701112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904e418-31ca-4d0b-a3b7-8a4ef439e575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1179882152342bcbbd62aca69dccfcc, user_id=6ea04e1ba0584f04a1e19c46821f3eef, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.703792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3904e418-31ca-4d0b-a3b7-8a4ef439e575 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.713779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1321/2657] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:47 2022] PUT /identity/v3/projects/b1179882152342bcbbd62aca69dccfcc/users/6ea04e1ba0584f04a1e19c46821f3eef/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:11:47.723633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc81409-d9a4-4b18-92e2-9810a6104e78 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.724098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc81409-d9a4-4b18-92e2-9810a6104e78 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.724203 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc81409-d9a4-4b18-92e2-9810a6104e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.751001 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-edc81409-d9a4-4b18-92e2-9810a6104e78 None None] MFA Rules not processed for user `6ea04e1ba0584f04a1e19c46821f3eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:47.752221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-edc81409-d9a4-4b18-92e2-9810a6104e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:47.832412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1337/2658] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:47.844850 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904e418-31ca-4d0b-a3b7-8a4ef439e575 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:47.846614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3904e418-31ca-4d0b-a3b7-8a4ef439e575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:47.904102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3904e418-31ca-4d0b-a3b7-8a4ef439e575 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:47.907786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f43c33-ce71-4afe-a0a3-ce49a7eb2f14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:47.908440 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f43c33-ce71-4afe-a0a3-ce49a7eb2f14 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:47.908724 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f43c33-ce71-4afe-a0a3-ce49a7eb2f14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:47.909000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f43c33-ce71-4afe-a0a3-ce49a7eb2f14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:47.918567 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f43c33-ce71-4afe-a0a3-ce49a7eb2f14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:47.920016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f43c33-ce71-4afe-a0a3-ce49a7eb2f14 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:47.930232 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1322/2659] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:48.029910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc81409-d9a4-4b18-92e2-9810a6104e78 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:48.040937 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc81409-d9a4-4b18-92e2-9810a6104e78 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:48.044887 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cdcee-8b6d-406d-bc20-f3a922264b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:48.045533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cdcee-8b6d-406d-bc20-f3a922264b2c service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:48.045830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cdcee-8b6d-406d-bc20-f3a922264b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:48.046106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151cdcee-8b6d-406d-bc20-f3a922264b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:48.103454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cdcee-8b6d-406d-bc20-f3a922264b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:48.104797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151cdcee-8b6d-406d-bc20-f3a922264b2c service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:48.114508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1338/2660] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:53.950251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddda5c-9aef-4a94-9bd8-81df214703fe None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:53.950928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddda5c-9aef-4a94-9bd8-81df214703fe None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:53.950928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddda5c-9aef-4a94-9bd8-81df214703fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:53.968657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ddddda5c-9aef-4a94-9bd8-81df214703fe None None] MFA Rules not processed for user `4ad8d2d3e04b4e06bff2c64bff5a1ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:53.969870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ddddda5c-9aef-4a94-9bd8-81df214703fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:53.974343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:54.027452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1323/2661] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:54.918409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cdcee-8b6d-406d-bc20-f3a922264b2c service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:54.946364 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=100 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:54.986439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151cdcee-8b6d-406d-bc20-f3a922264b2c service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:54.991951 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:54.992827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:54.993111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:54.993398 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] PATH_INFO: `/v3/users/6510447467514897bd9c4802d38d4a22` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:54.994633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6510447467514897bd9c4802d38d4a22)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:54.996649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:55.043884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6510447467514897bd9c4802d38d4a22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.055868 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6510447467514897bd9c4802d38d4a22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.063156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6510447467514897bd9c4802d38d4a22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.063874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6510447467514897bd9c4802d38d4a22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.070621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6510447467514897bd9c4802d38d4a22'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.076428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1339/2662] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:54 2022] DELETE /identity/v3/users/6510447467514897bd9c4802d38d4a22 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:55.080882 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ae26f-d9a9-4401-b87e-3f6ff6775e78 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:55.081369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ae26f-d9a9-4401-b87e-3f6ff6775e78 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:55.081552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ae26f-d9a9-4401-b87e-3f6ff6775e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:55.100016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-936ae26f-d9a9-4401-b87e-3f6ff6775e78 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:55.101246 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-936ae26f-d9a9-4401-b87e-3f6ff6775e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:55.163321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1324/2663] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:55.172981 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:55.174542 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:55.236750 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cca9e9-7ce0-49ce-85de-24a72eb5277f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:55.240717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f93f0b-e9ed-4c30-80b0-f8b9fc4ef201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:55.241363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f93f0b-e9ed-4c30-80b0-f8b9fc4ef201 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:55.241694 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f93f0b-e9ed-4c30-80b0-f8b9fc4ef201 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:55.241916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f93f0b-e9ed-4c30-80b0-f8b9fc4ef201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:55.250965 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f93f0b-e9ed-4c30-80b0-f8b9fc4ef201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:55.252421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f93f0b-e9ed-4c30-80b0-f8b9fc4ef201 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:55.267456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1340/2664] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:55.546218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ae26f-d9a9-4401-b87e-3f6ff6775e78 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:55.548200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-936ae26f-d9a9-4401-b87e-3f6ff6775e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:55.561117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ae26f-d9a9-4401-b87e-3f6ff6775e78 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:55.565889 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:55.567294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:55.567584 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:55.567860 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] PATH_INFO: `/v3/projects/fbced3dddcc545e4aee8c98e6c8b0a45` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:55.569240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbced3dddcc545e4aee8c98e6c8b0a45)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:55.571203 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:55.619651 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbced3dddcc545e4aee8c98e6c8b0a45', 'request_id': 'req-3f7a406c-732c-4a31-a33d-1bd961f056a9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.626097 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbced3dddcc545e4aee8c98e6c8b0a45', 'request_id': 'req-3f7a406c-732c-4a31-a33d-1bd961f056a9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.633005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbced3dddcc545e4aee8c98e6c8b0a45', 'request_id': 'req-3f7a406c-732c-4a31-a33d-1bd961f056a9'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.645938 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] The token cache is being invalidate because project fbced3dddcc545e4aee8c98e6c8b0a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:55.646224 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.646865 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.648412 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1325/2665] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:55 2022] DELETE /identity/v3/projects/fbced3dddcc545e4aee8c98e6c8b0a45 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:55.652509 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f93f0b-e9ed-4c30-80b0-f8b9fc4ef201 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:55.662111 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f93f0b-e9ed-4c30-80b0-f8b9fc4ef201 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:55.709665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:55.725784 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:55.726681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:55.726969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:55.727261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] PATH_INFO: `/v3/users/6ea04e1ba0584f04a1e19c46821f3eef` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:55.728474 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ea04e1ba0584f04a1e19c46821f3eef)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:55.730476 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:55.786848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ea04e1ba0584f04a1e19c46821f3eef'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.799935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ea04e1ba0584f04a1e19c46821f3eef'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.809310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ea04e1ba0584f04a1e19c46821f3eef'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.810065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ea04e1ba0584f04a1e19c46821f3eef'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.820029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-2cb294b1-2abe-4f4d-b33d-6ff0842bfcc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ea04e1ba0584f04a1e19c46821f3eef'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:55.826005 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1341/2666] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:11:55 2022] DELETE /identity/v3/users/6ea04e1ba0584f04a1e19c46821f3eef => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:56.098248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:56.100317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:56.113248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a406c-732c-4a31-a33d-1bd961f056a9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:56.149100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:56.173359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:56.174216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.174538 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.174820 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] PATH_INFO: `/v3/projects/b1179882152342bcbbd62aca69dccfcc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.176645 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1179882152342bcbbd62aca69dccfcc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:56.178616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:56.223384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1179882152342bcbbd62aca69dccfcc', 'request_id': 'req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:56.228839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1179882152342bcbbd62aca69dccfcc', 'request_id': 'req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:56.234895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1179882152342bcbbd62aca69dccfcc', 'request_id': 'req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:56.247505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] The token cache is being invalidate because project b1179882152342bcbbd62aca69dccfcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:11:56.247788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:56.248400 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9586ce25-41e8-4b70-8a5e-4aa3bb550aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:11:56.250024 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1326/2667] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:11:56 2022] DELETE /identity/v3/projects/b1179882152342bcbbd62aca69dccfcc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:56.254952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f32e71-2f2f-4812-8911-72854f9a8864 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.255477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f32e71-2f2f-4812-8911-72854f9a8864 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.255477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f32e71-2f2f-4812-8911-72854f9a8864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.272457 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b1f32e71-2f2f-4812-8911-72854f9a8864 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:56.273644 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b1f32e71-2f2f-4812-8911-72854f9a8864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:56.330597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1342/2668] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:56.338179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb433f-34fa-452a-b146-fdfe60d75e20 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.338609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb433f-34fa-452a-b146-fdfe60d75e20 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.338786 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb433f-34fa-452a-b146-fdfe60d75e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.360460 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-21eb433f-34fa-452a-b146-fdfe60d75e20 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:56.362514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-21eb433f-34fa-452a-b146-fdfe60d75e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:56.378500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1327/2669] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:56.383044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f32e71-2f2f-4812-8911-72854f9a8864 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:56.396563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f32e71-2f2f-4812-8911-72854f9a8864 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:56.403458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc8566-a4f5-4535-b45d-43eaae2c643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:56.404454 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc8566-a4f5-4535-b45d-43eaae2c643e admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.404886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc8566-a4f5-4535-b45d-43eaae2c643e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.405304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc8566-a4f5-4535-b45d-43eaae2c643e admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.406421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc8566-a4f5-4535-b45d-43eaae2c643e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:56.406941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc8566-a4f5-4535-b45d-43eaae2c643e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:56.409544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc8566-a4f5-4535-b45d-43eaae2c643e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:56.427874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1343/2670] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:11:56.432013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb433f-34fa-452a-b146-fdfe60d75e20 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:56.441462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb433f-34fa-452a-b146-fdfe60d75e20 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:56.446117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777603ca-36b0-47c8-b917-c7960ac1ded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:56.446800 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777603ca-36b0-47c8-b917-c7960ac1ded9 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.447086 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777603ca-36b0-47c8-b917-c7960ac1ded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.447367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777603ca-36b0-47c8-b917-c7960ac1ded9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.448092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777603ca-36b0-47c8-b917-c7960ac1ded9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:56.450001 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777603ca-36b0-47c8-b917-c7960ac1ded9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:56.476123 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1328/2671] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:11:56 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:11:56.480270 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc8566-a4f5-4535-b45d-43eaae2c643e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:56.491040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc8566-a4f5-4535-b45d-43eaae2c643e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:56.537868 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:56.548024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fa58e-02a2-4c58-a0d4-50aa2b6b7ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:56.548690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fa58e-02a2-4c58-a0d4-50aa2b6b7ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.549015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fa58e-02a2-4c58-a0d4-50aa2b6b7ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.549299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fa58e-02a2-4c58-a0d4-50aa2b6b7ac8 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.549991 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fa58e-02a2-4c58-a0d4-50aa2b6b7ac8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:56.551961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8fa58e-02a2-4c58-a0d4-50aa2b6b7ac8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:56.570062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1344/2672] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:11:56 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:56.574175 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777603ca-36b0-47c8-b917-c7960ac1ded9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:56.584953 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777603ca-36b0-47c8-b917-c7960ac1ded9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:56.589563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d2649-6176-4202-a3a5-22e9d9a74f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:56.590210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d2649-6176-4202-a3a5-22e9d9a74f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.590551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d2649-6176-4202-a3a5-22e9d9a74f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.590836 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d2649-6176-4202-a3a5-22e9d9a74f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.591559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d2649-6176-4202-a3a5-22e9d9a74f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:56.593170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d2649-6176-4202-a3a5-22e9d9a74f11 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:56.604936 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1329/2673] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:56.608848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fa58e-02a2-4c58-a0d4-50aa2b6b7ac8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:56.618938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fa58e-02a2-4c58-a0d4-50aa2b6b7ac8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:56.623507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34661981-32e9-4b7c-8f0d-226367daab73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:56.624150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34661981-32e9-4b7c-8f0d-226367daab73 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.624433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34661981-32e9-4b7c-8f0d-226367daab73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.624706 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34661981-32e9-4b7c-8f0d-226367daab73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.625466 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34661981-32e9-4b7c-8f0d-226367daab73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:56.627075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34661981-32e9-4b7c-8f0d-226367daab73 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:56.636824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1345/2674] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:56.640793 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d2649-6176-4202-a3a5-22e9d9a74f11 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:56.652860 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d2649-6176-4202-a3a5-22e9d9a74f11 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:56.657522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231ed1d-b523-4a89-b5ba-10db081270d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:56.658379 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231ed1d-b523-4a89-b5ba-10db081270d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.658690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231ed1d-b523-4a89-b5ba-10db081270d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.658963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231ed1d-b523-4a89-b5ba-10db081270d7 admin admin] PATH_INFO: `/v3/projects/cc72aedcd1094d33a4c1154437aeda34/users/642eb0d4a8fd4d1183a1f7ac78f36a55/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.675157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231ed1d-b523-4a89-b5ba-10db081270d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc72aedcd1094d33a4c1154437aeda34, user_id=642eb0d4a8fd4d1183a1f7ac78f36a55, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:56.677815 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231ed1d-b523-4a89-b5ba-10db081270d7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:56.693594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1330/2675] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:11:56 2022] PUT /identity/v3/projects/cc72aedcd1094d33a4c1154437aeda34/users/642eb0d4a8fd4d1183a1f7ac78f36a55/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:56.704872 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717fb2e-a311-42ed-af05-6a08d6265f0a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.705168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717fb2e-a311-42ed-af05-6a08d6265f0a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.705455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717fb2e-a311-42ed-af05-6a08d6265f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.733917 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6717fb2e-a311-42ed-af05-6a08d6265f0a None None] MFA Rules not processed for user `642eb0d4a8fd4d1183a1f7ac78f36a55`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:56.735161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6717fb2e-a311-42ed-af05-6a08d6265f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:56.799057 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1346/2676] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:11:56.810344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231ed1d-b523-4a89-b5ba-10db081270d7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:56.812206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9231ed1d-b523-4a89-b5ba-10db081270d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:56.878635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231ed1d-b523-4a89-b5ba-10db081270d7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:56.882402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f388e3-8191-4a42-aec1-28fb2b5fd061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:56.883084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f388e3-8191-4a42-aec1-28fb2b5fd061 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:56.883372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f388e3-8191-4a42-aec1-28fb2b5fd061 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:56.883655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f388e3-8191-4a42-aec1-28fb2b5fd061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:56.892080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f388e3-8191-4a42-aec1-28fb2b5fd061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:56.893522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f388e3-8191-4a42-aec1-28fb2b5fd061 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:56.902364 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1331/2677] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:11:58.574565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717fb2e-a311-42ed-af05-6a08d6265f0a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:58.575987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6717fb2e-a311-42ed-af05-6a08d6265f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:58.640772 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717fb2e-a311-42ed-af05-6a08d6265f0a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:58.644373 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d8293-7853-4149-a79e-631b48eb77d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:58.645067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d8293-7853-4149-a79e-631b48eb77d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:58.645350 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d8293-7853-4149-a79e-631b48eb77d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:58.645627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d8293-7853-4149-a79e-631b48eb77d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:58.647526 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7c2d8293-7853-4149-a79e-631b48eb77d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:58.711058 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d8293-7853-4149-a79e-631b48eb77d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:58.712539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d8293-7853-4149-a79e-631b48eb77d9 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:58.722056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1347/2678] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:11:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:02.110768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f388e3-8191-4a42-aec1-28fb2b5fd061 service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:02.112241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e5f388e3-8191-4a42-aec1-28fb2b5fd061 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:02.135843 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f388e3-8191-4a42-aec1-28fb2b5fd061 service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:02.141425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:02.143118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:02.143345 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:02.143627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] PATH_INFO: `/v3/users/4e6db41ec3b94c27bb4af92a35ee0e6d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:02.145098 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e6db41ec3b94c27bb4af92a35ee0e6d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:02.147971 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:02.149690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:02.225484 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e6db41ec3b94c27bb4af92a35ee0e6d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:02.226191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e6db41ec3b94c27bb4af92a35ee0e6d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:02.234303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e6db41ec3b94c27bb4af92a35ee0e6d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:02.234959 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e6db41ec3b94c27bb4af92a35ee0e6d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:02.248315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c83900bd-55ff-45fb-83ae-3620394cdb11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e6db41ec3b94c27bb4af92a35ee0e6d'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:02.261318 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1332/2679] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:12:02 2022] DELETE /identity/v3/users/4e6db41ec3b94c27bb4af92a35ee0e6d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:02.619455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d8293-7853-4149-a79e-631b48eb77d9 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:02.638884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d8293-7853-4149-a79e-631b48eb77d9 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:02.650596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:02.705836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:02.706727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:02.707527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:02.708417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] PATH_INFO: `/v3/projects/aaa9e1a61984467ab8efae451b270558` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:02.709681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa9e1a61984467ab8efae451b270558)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:02.712380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:02.766868 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa9e1a61984467ab8efae451b270558', 'request_id': 'req-33dc82bc-5d80-4041-864c-71fe43d85a7a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:02.773695 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa9e1a61984467ab8efae451b270558', 'request_id': 'req-33dc82bc-5d80-4041-864c-71fe43d85a7a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:02.779236 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa9e1a61984467ab8efae451b270558', 'request_id': 'req-33dc82bc-5d80-4041-864c-71fe43d85a7a'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:02.792838 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] The token cache is being invalidate because project aaa9e1a61984467ab8efae451b270558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:12:02.793206 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:02.793825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-33dc82bc-5d80-4041-864c-71fe43d85a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:02.795559 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1348/2680] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:12:02 2022] DELETE /identity/v3/projects/aaa9e1a61984467ab8efae451b270558 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:02.800527 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce48f4-0775-467c-8dbb-819b0d1400c4 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:02.800845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce48f4-0775-467c-8dbb-819b0d1400c4 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:02.801178 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce48f4-0775-467c-8dbb-819b0d1400c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:02.822396 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-3dce48f4-0775-467c-8dbb-819b0d1400c4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:02.823617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-3dce48f4-0775-467c-8dbb-819b0d1400c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:02.886936 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1333/2681] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:12:02.895402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade2e50-35c8-4c52-a971-b2048a0ca9e9 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:02.895694 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade2e50-35c8-4c52-a971-b2048a0ca9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:02.895973 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade2e50-35c8-4c52-a971-b2048a0ca9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:02.915354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-5ade2e50-35c8-4c52-a971-b2048a0ca9e9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:02.916839 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5ade2e50-35c8-4c52-a971-b2048a0ca9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:02.925517 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1349/2682] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:02.929619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce48f4-0775-467c-8dbb-819b0d1400c4 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:02.939459 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce48f4-0775-467c-8dbb-819b0d1400c4 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:02.944543 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f7e0-77a6-4d5a-b04f-2b76c645387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:02.945198 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f7e0-77a6-4d5a-b04f-2b76c645387c admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:02.945486 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f7e0-77a6-4d5a-b04f-2b76c645387c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:02.945809 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f7e0-77a6-4d5a-b04f-2b76c645387c admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:02.946519 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f7e0-77a6-4d5a-b04f-2b76c645387c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:12:02.946831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f7e0-77a6-4d5a-b04f-2b76c645387c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:02.948424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f7e0-77a6-4d5a-b04f-2b76c645387c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:02.964193 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1334/2683] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:12:02.965617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade2e50-35c8-4c52-a971-b2048a0ca9e9 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:02.975197 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade2e50-35c8-4c52-a971-b2048a0ca9e9 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:02.979855 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b38be-967a-4992-b2d5-97f262f08e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:02.980504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b38be-967a-4992-b2d5-97f262f08e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:02.980786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b38be-967a-4992-b2d5-97f262f08e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:02.981066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b38be-967a-4992-b2d5-97f262f08e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:02.981761 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b38be-967a-4992-b2d5-97f262f08e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:02.983665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b38be-967a-4992-b2d5-97f262f08e45 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:03.014482 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1350/2684] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:12:02 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:03.018206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f7e0-77a6-4d5a-b04f-2b76c645387c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:03.031303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f7e0-77a6-4d5a-b04f-2b76c645387c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:03.100032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650386d1-3245-47e4-b800-1b06b66ae158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:03.100706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650386d1-3245-47e4-b800-1b06b66ae158 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:03.100994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650386d1-3245-47e4-b800-1b06b66ae158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:03.101278 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650386d1-3245-47e4-b800-1b06b66ae158 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:03.102166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650386d1-3245-47e4-b800-1b06b66ae158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:03.104154 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650386d1-3245-47e4-b800-1b06b66ae158 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:03.115026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:03.134537 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1335/2685] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:12:03 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:03.138626 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b38be-967a-4992-b2d5-97f262f08e45 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:03.149541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b38be-967a-4992-b2d5-97f262f08e45 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:03.154311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abf062-47f1-4c5e-84dd-bed2d44a1192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:03.155025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abf062-47f1-4c5e-84dd-bed2d44a1192 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:03.155311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abf062-47f1-4c5e-84dd-bed2d44a1192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:03.155595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abf062-47f1-4c5e-84dd-bed2d44a1192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:03.156326 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abf062-47f1-4c5e-84dd-bed2d44a1192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:03.158040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abf062-47f1-4c5e-84dd-bed2d44a1192 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:03.170506 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1351/2686] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:12:03.177256 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650386d1-3245-47e4-b800-1b06b66ae158 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:03.188885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650386d1-3245-47e4-b800-1b06b66ae158 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:03.194026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa25ce-fe9f-431c-845a-9dd1533d6310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:03.194734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa25ce-fe9f-431c-845a-9dd1533d6310 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:03.195019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa25ce-fe9f-431c-845a-9dd1533d6310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:03.195297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa25ce-fe9f-431c-845a-9dd1533d6310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:03.196016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa25ce-fe9f-431c-845a-9dd1533d6310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:03.197611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa25ce-fe9f-431c-845a-9dd1533d6310 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:03.210474 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1336/2687] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:03.212425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abf062-47f1-4c5e-84dd-bed2d44a1192 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:03.222213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abf062-47f1-4c5e-84dd-bed2d44a1192 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:03.226862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261bcbd-c724-4572-9e2c-b6acd8ab5dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:03.227749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261bcbd-c724-4572-9e2c-b6acd8ab5dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:03.228033 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261bcbd-c724-4572-9e2c-b6acd8ab5dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:03.228309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261bcbd-c724-4572-9e2c-b6acd8ab5dd0 admin admin] PATH_INFO: `/v3/projects/6a8f75c4a05f4676b9f2bd4a70ad2f85/users/aa06d1117175407d98da4c78b78c8ac6/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:03.244004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261bcbd-c724-4572-9e2c-b6acd8ab5dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8f75c4a05f4676b9f2bd4a70ad2f85, user_id=aa06d1117175407d98da4c78b78c8ac6, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:03.246758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261bcbd-c724-4572-9e2c-b6acd8ab5dd0 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:03.261500 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1352/2688] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:12:03 2022] PUT /identity/v3/projects/6a8f75c4a05f4676b9f2bd4a70ad2f85/users/aa06d1117175407d98da4c78b78c8ac6/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:03.272408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ccee4-ba3d-4c9e-b397-5f7241f81723 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:03.272709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ccee4-ba3d-4c9e-b397-5f7241f81723 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:03.273020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ccee4-ba3d-4c9e-b397-5f7241f81723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:03.304190 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-250ccee4-ba3d-4c9e-b397-5f7241f81723 None None] MFA Rules not processed for user `aa06d1117175407d98da4c78b78c8ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:03.305690 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-250ccee4-ba3d-4c9e-b397-5f7241f81723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:03.373894 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1337/2689] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:03.385199 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261bcbd-c724-4572-9e2c-b6acd8ab5dd0 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:03.386964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-2261bcbd-c724-4572-9e2c-b6acd8ab5dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:03.428208 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:03.445578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261bcbd-c724-4572-9e2c-b6acd8ab5dd0 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:03.449539 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e3c84-1d93-4898-8157-2c49e4869b42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:03.450180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3c84-1d93-4898-8157-2c49e4869b42 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:03.450544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3c84-1d93-4898-8157-2c49e4869b42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:03.450829 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e3c84-1d93-4898-8157-2c49e4869b42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:03.460397 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e3c84-1d93-4898-8157-2c49e4869b42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:03.461875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e3c84-1d93-4898-8157-2c49e4869b42 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:03.470671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1353/2690] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:03.490830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ccee4-ba3d-4c9e-b397-5f7241f81723 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:03.552137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ccee4-ba3d-4c9e-b397-5f7241f81723 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:03.556711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:03.557560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:03.557846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:03.558124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] PATH_INFO: `/v3/users/aa06d1117175407d98da4c78b78c8ac6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:03.559357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa06d1117175407d98da4c78b78c8ac6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:03.561287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:03.605884 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa06d1117175407d98da4c78b78c8ac6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:03.606508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa06d1117175407d98da4c78b78c8ac6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:03.613615 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa06d1117175407d98da4c78b78c8ac6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:03.614676 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa06d1117175407d98da4c78b78c8ac6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:03.626929 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa06d1117175407d98da4c78b78c8ac6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:03.631996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:03.643010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1338/2691] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:12:03 2022] DELETE /identity/v3/users/aa06d1117175407d98da4c78b78c8ac6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:03.649306 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b362d7-c8fe-4757-ab64-6a3920ca2707 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:03.649812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b362d7-c8fe-4757-ab64-6a3920ca2707 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:03.649957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b362d7-c8fe-4757-ab64-6a3920ca2707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:03.667249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-15b362d7-c8fe-4757-ab64-6a3920ca2707 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:03.668632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-15b362d7-c8fe-4757-ab64-6a3920ca2707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:03.731381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1354/2692] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:03.741241 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:03.742679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:03.800601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4368cab-a23d-4b15-b7a9-3f5922b405f6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:03.804309 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0c52e-3d9e-48d4-8ba8-bbc7ac6ffc77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:03.804962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0c52e-3d9e-48d4-8ba8-bbc7ac6ffc77 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:03.805252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0c52e-3d9e-48d4-8ba8-bbc7ac6ffc77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:03.805544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0c52e-3d9e-48d4-8ba8-bbc7ac6ffc77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:03.814934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0c52e-3d9e-48d4-8ba8-bbc7ac6ffc77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:03.816873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0c52e-3d9e-48d4-8ba8-bbc7ac6ffc77 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:03.826425 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1339/2693] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:04.097939 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b362d7-c8fe-4757-ab64-6a3920ca2707 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:04.099384 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-15b362d7-c8fe-4757-ab64-6a3920ca2707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:04.115411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b362d7-c8fe-4757-ab64-6a3920ca2707 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:04.120045 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:04.120916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.121214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.121475 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] PATH_INFO: `/v3/projects/6a8f75c4a05f4676b9f2bd4a70ad2f85` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.122773 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8f75c4a05f4676b9f2bd4a70ad2f85)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:04.124718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:04.171624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8f75c4a05f4676b9f2bd4a70ad2f85', 'request_id': 'req-b5719913-845b-4f70-9eb8-71f1fdf85b80'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:04.178352 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8f75c4a05f4676b9f2bd4a70ad2f85', 'request_id': 'req-b5719913-845b-4f70-9eb8-71f1fdf85b80'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:04.184961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8f75c4a05f4676b9f2bd4a70ad2f85', 'request_id': 'req-b5719913-845b-4f70-9eb8-71f1fdf85b80'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:04.201423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] The token cache is being invalidate because project 6a8f75c4a05f4676b9f2bd4a70ad2f85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:12:04.201746 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:04.202387 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-b5719913-845b-4f70-9eb8-71f1fdf85b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:04.204216 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1355/2694] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:12:04 2022] DELETE /identity/v3/projects/6a8f75c4a05f4676b9f2bd4a70ad2f85 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:04.208861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8fa22-4d64-4971-b700-42f58526f02c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.209341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8fa22-4d64-4971-b700-42f58526f02c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.209456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8fa22-4d64-4971-b700-42f58526f02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.227376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f7f8fa22-4d64-4971-b700-42f58526f02c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:04.230631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f7f8fa22-4d64-4971-b700-42f58526f02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:04.305084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1340/2695] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:04.313420 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3fb7b-e076-4111-a6e5-8a6e73f3c3e4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.313767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3fb7b-e076-4111-a6e5-8a6e73f3c3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.314054 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3fb7b-e076-4111-a6e5-8a6e73f3c3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.334982 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-52d3fb7b-e076-4111-a6e5-8a6e73f3c3e4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:04.336232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-52d3fb7b-e076-4111-a6e5-8a6e73f3c3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:04.344925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1356/2696] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:04.349730 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8fa22-4d64-4971-b700-42f58526f02c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:04.372251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8fa22-4d64-4971-b700-42f58526f02c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:04.377445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e694673-0d71-4f0b-b5f2-fa43bd9d029d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:04.378114 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e694673-0d71-4f0b-b5f2-fa43bd9d029d admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.378398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e694673-0d71-4f0b-b5f2-fa43bd9d029d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.378750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e694673-0d71-4f0b-b5f2-fa43bd9d029d admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.379739 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e694673-0d71-4f0b-b5f2-fa43bd9d029d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:12:04.380057 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e694673-0d71-4f0b-b5f2-fa43bd9d029d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:04.381696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e694673-0d71-4f0b-b5f2-fa43bd9d029d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:04.397000 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1341/2697] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:12:04.398066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3fb7b-e076-4111-a6e5-8a6e73f3c3e4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:04.407229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3fb7b-e076-4111-a6e5-8a6e73f3c3e4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:04.412392 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d4ed5-c5dc-4579-bbec-788fda190bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:04.413038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d4ed5-c5dc-4579-bbec-788fda190bba admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.413327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d4ed5-c5dc-4579-bbec-788fda190bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.413602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51d4ed5-c5dc-4579-bbec-788fda190bba admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.414294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d4ed5-c5dc-4579-bbec-788fda190bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:04.416888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51d4ed5-c5dc-4579-bbec-788fda190bba admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:04.447467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1357/2698] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:12:04 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:04.451624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e694673-0d71-4f0b-b5f2-fa43bd9d029d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:04.462176 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e694673-0d71-4f0b-b5f2-fa43bd9d029d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:04.524039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4583f-0527-4d0f-a099-8df998092787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:04.524703 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4583f-0527-4d0f-a099-8df998092787 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.524983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4583f-0527-4d0f-a099-8df998092787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.525257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4583f-0527-4d0f-a099-8df998092787 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.525970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4583f-0527-4d0f-a099-8df998092787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:04.527915 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a4583f-0527-4d0f-a099-8df998092787 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:04.547855 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1342/2699] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:12:04 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:04.552753 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d4ed5-c5dc-4579-bbec-788fda190bba admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:04.563557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51d4ed5-c5dc-4579-bbec-788fda190bba admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:04.568545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27f735-9bc1-41c1-806f-703fe1b69a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:04.569241 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27f735-9bc1-41c1-806f-703fe1b69a4f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.569533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27f735-9bc1-41c1-806f-703fe1b69a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.569812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27f735-9bc1-41c1-806f-703fe1b69a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.570612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27f735-9bc1-41c1-806f-703fe1b69a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:04.572216 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27f735-9bc1-41c1-806f-703fe1b69a4f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:04.583225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1358/2700] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:04.587108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4583f-0527-4d0f-a099-8df998092787 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:04.596873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4583f-0527-4d0f-a099-8df998092787 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:04.602550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32db51c-4d63-49ee-8abc-358a4ee2b85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:04.603254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32db51c-4d63-49ee-8abc-358a4ee2b85e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.603591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32db51c-4d63-49ee-8abc-358a4ee2b85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.603873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32db51c-4d63-49ee-8abc-358a4ee2b85e admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.604591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32db51c-4d63-49ee-8abc-358a4ee2b85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:04.606251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32db51c-4d63-49ee-8abc-358a4ee2b85e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:04.617226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1343/2701] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:04.621654 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27f735-9bc1-41c1-806f-703fe1b69a4f admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:04.631808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27f735-9bc1-41c1-806f-703fe1b69a4f admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:04.636895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0798e0-45a2-4760-918b-7cf9daf5d04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:04.637820 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0798e0-45a2-4760-918b-7cf9daf5d04d admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.638112 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0798e0-45a2-4760-918b-7cf9daf5d04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.638386 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0798e0-45a2-4760-918b-7cf9daf5d04d admin admin] PATH_INFO: `/v3/projects/8e3b2848096f437fada2e0c8a0d09e0e/users/27d3c22c7f3b482d9e47910d7bef2eba/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.653746 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0798e0-45a2-4760-918b-7cf9daf5d04d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3b2848096f437fada2e0c8a0d09e0e, user_id=27d3c22c7f3b482d9e47910d7bef2eba, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:04.656579 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0798e0-45a2-4760-918b-7cf9daf5d04d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:04.665315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:04.671449 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1359/2702] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:12:04 2022] PUT /identity/v3/projects/8e3b2848096f437fada2e0c8a0d09e0e/users/27d3c22c7f3b482d9e47910d7bef2eba/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:04.683033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d2bc4-9cd7-4468-b358-4ea22c08350a None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.683374 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d2bc4-9cd7-4468-b358-4ea22c08350a None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.683607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d2bc4-9cd7-4468-b358-4ea22c08350a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.691130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:04.713012 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-e67d2bc4-9cd7-4468-b358-4ea22c08350a None None] MFA Rules not processed for user `27d3c22c7f3b482d9e47910d7bef2eba`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:04.714221 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e67d2bc4-9cd7-4468-b358-4ea22c08350a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:04.788039 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1344/2703] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:04.793229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0798e0-45a2-4760-918b-7cf9daf5d04d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:04.854755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0798e0-45a2-4760-918b-7cf9daf5d04d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:04.859372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90378b-6c1e-42b7-a3da-04b2787a6afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:04.860025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90378b-6c1e-42b7-a3da-04b2787a6afd admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.860314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90378b-6c1e-42b7-a3da-04b2787a6afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.860595 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90378b-6c1e-42b7-a3da-04b2787a6afd admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.861274 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90378b-6c1e-42b7-a3da-04b2787a6afd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:04.863277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90378b-6c1e-42b7-a3da-04b2787a6afd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:04.887668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1360/2704] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:12:04 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:12:04.891753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d2bc4-9cd7-4468-b358-4ea22c08350a None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:04.902685 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d2bc4-9cd7-4468-b358-4ea22c08350a None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:04.959402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602cc4e-cfc0-481e-8473-caf0e72b9bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:04.960022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602cc4e-cfc0-481e-8473-caf0e72b9bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:04.960308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602cc4e-cfc0-481e-8473-caf0e72b9bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:04.960586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602cc4e-cfc0-481e-8473-caf0e72b9bf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:04.961280 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602cc4e-cfc0-481e-8473-caf0e72b9bf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:04.963277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602cc4e-cfc0-481e-8473-caf0e72b9bf8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:04.982598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1345/2705] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:12:04 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:04.986667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90378b-6c1e-42b7-a3da-04b2787a6afd admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:04.997308 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90378b-6c1e-42b7-a3da-04b2787a6afd admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:05.001972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc5855-baec-4516-b3b7-d2951bf14927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:05.002655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc5855-baec-4516-b3b7-d2951bf14927 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.002941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc5855-baec-4516-b3b7-d2951bf14927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.003217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc5855-baec-4516-b3b7-d2951bf14927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.003961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc5855-baec-4516-b3b7-d2951bf14927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:05.005543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc5855-baec-4516-b3b7-d2951bf14927 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:05.016741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1361/2706] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:05.021235 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602cc4e-cfc0-481e-8473-caf0e72b9bf8 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:05.031340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602cc4e-cfc0-481e-8473-caf0e72b9bf8 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:05.036585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786787-26bb-4a5e-8662-98be0ce8961e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:05.037468 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786787-26bb-4a5e-8662-98be0ce8961e admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.037753 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786787-26bb-4a5e-8662-98be0ce8961e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.038040 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786787-26bb-4a5e-8662-98be0ce8961e admin admin] PATH_INFO: `/v3/projects/f4c635fe56ef49e9be1ba44f6c90c858/users/0611753741f7490982b73865a33c655f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.053349 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f786787-26bb-4a5e-8662-98be0ce8961e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c635fe56ef49e9be1ba44f6c90c858, user_id=0611753741f7490982b73865a33c655f, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:05.057038 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f786787-26bb-4a5e-8662-98be0ce8961e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:05.069914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1346/2707] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:12:05 2022] PUT /identity/v3/projects/f4c635fe56ef49e9be1ba44f6c90c858/users/0611753741f7490982b73865a33c655f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:05.073876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc5855-baec-4516-b3b7-d2951bf14927 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:05.084388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc5855-baec-4516-b3b7-d2951bf14927 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:05.139530 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18a195-ac3f-4cf3-b4d6-df289555a164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:05.140188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a195-ac3f-4cf3-b4d6-df289555a164 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.140471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a195-ac3f-4cf3-b4d6-df289555a164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.140749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a195-ac3f-4cf3-b4d6-df289555a164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.141461 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18a195-ac3f-4cf3-b4d6-df289555a164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:05.143090 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18a195-ac3f-4cf3-b4d6-df289555a164 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:05.153239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1362/2708] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:05.157310 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786787-26bb-4a5e-8662-98be0ce8961e admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:05.167461 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786787-26bb-4a5e-8662-98be0ce8961e admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:05.171995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea821b-4680-4012-99d4-df7dc248320b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:05.172866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea821b-4680-4012-99d4-df7dc248320b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.173155 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea821b-4680-4012-99d4-df7dc248320b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.173452 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea821b-4680-4012-99d4-df7dc248320b admin admin] PATH_INFO: `/v3/projects/f4c635fe56ef49e9be1ba44f6c90c858/users/0611753741f7490982b73865a33c655f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.175542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea821b-4680-4012-99d4-df7dc248320b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c635fe56ef49e9be1ba44f6c90c858, user_id=0611753741f7490982b73865a33c655f, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:05.178215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea821b-4680-4012-99d4-df7dc248320b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:05.188255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1347/2709] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:12:05 2022] PUT /identity/v3/projects/f4c635fe56ef49e9be1ba44f6c90c858/users/0611753741f7490982b73865a33c655f/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:05.200072 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc06313-5dee-4f3a-9b10-cf346857e5bc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.200374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc06313-5dee-4f3a-9b10-cf346857e5bc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.200684 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc06313-5dee-4f3a-9b10-cf346857e5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.227162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6cc06313-5dee-4f3a-9b10-cf346857e5bc None None] MFA Rules not processed for user `0611753741f7490982b73865a33c655f`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:05.228307 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6cc06313-5dee-4f3a-9b10-cf346857e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:05.286714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:05.296211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1363/2710] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:05.306738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea821b-4680-4012-99d4-df7dc248320b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:05.308472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-35ea821b-4680-4012-99d4-df7dc248320b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:05.354738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:05.370525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea821b-4680-4012-99d4-df7dc248320b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:05.375092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919827c-8b41-437e-b91f-af2fca28e81b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:05.375326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919827c-8b41-437e-b91f-af2fca28e81b service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.375611 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919827c-8b41-437e-b91f-af2fca28e81b service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.375947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919827c-8b41-437e-b91f-af2fca28e81b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.400867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc06313-5dee-4f3a-9b10-cf346857e5bc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:05.402315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6cc06313-5dee-4f3a-9b10-cf346857e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:05.440568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919827c-8b41-437e-b91f-af2fca28e81b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:05.442569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919827c-8b41-437e-b91f-af2fca28e81b service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:05.455806 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1348/2711] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:05.469219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc06313-5dee-4f3a-9b10-cf346857e5bc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:05.476144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:05.477142 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.477943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.477943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] PATH_INFO: `/v3/users/642eb0d4a8fd4d1183a1f7ac78f36a55` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.479478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=642eb0d4a8fd4d1183a1f7ac78f36a55)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:05.481568 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:05.519569 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919827c-8b41-437e-b91f-af2fca28e81b service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:05.521619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7919827c-8b41-437e-b91f-af2fca28e81b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:05.542560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919827c-8b41-437e-b91f-af2fca28e81b service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:05.546291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642eb0d4a8fd4d1183a1f7ac78f36a55'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:05.552218 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:05.553312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.553758 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.554639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] PATH_INFO: `/v3/users/c7a031ee64f446d081c7403c329ea7e7` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.556823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7a031ee64f446d081c7403c329ea7e7)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:05.560616 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:05.561129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642eb0d4a8fd4d1183a1f7ac78f36a55'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:05.571910 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642eb0d4a8fd4d1183a1f7ac78f36a55'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:05.572655 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642eb0d4a8fd4d1183a1f7ac78f36a55'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:05.582642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-206eea99-3c86-4dcb-9988-b9b8f72baf22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642eb0d4a8fd4d1183a1f7ac78f36a55'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:05.597856 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1364/2712] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:12:05 2022] DELETE /identity/v3/users/642eb0d4a8fd4d1183a1f7ac78f36a55 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:12:05.602310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb632a-f1b2-47f7-b8aa-aa149edc16ea None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.602664 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb632a-f1b2-47f7-b8aa-aa149edc16ea None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.602956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb632a-f1b2-47f7-b8aa-aa149edc16ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.632850 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b5fb632a-f1b2-47f7-b8aa-aa149edc16ea None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:05.633544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7a031ee64f446d081c7403c329ea7e7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:05.634315 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7a031ee64f446d081c7403c329ea7e7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:05.635419 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b5fb632a-f1b2-47f7-b8aa-aa149edc16ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:05.645243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7a031ee64f446d081c7403c329ea7e7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:05.645962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7a031ee64f446d081c7403c329ea7e7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:05.667757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-a1cc1a88-f017-40cf-b94c-69b132e3182b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7a031ee64f446d081c7403c329ea7e7'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:05.688338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1349/2713] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:12:05 2022] DELETE /identity/v3/users/c7a031ee64f446d081c7403c329ea7e7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:05.689962 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bac5-34cc-4ee6-94ab-a98c7231a332 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.692458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bac5-34cc-4ee6-94ab-a98c7231a332 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.692744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001bac5-34cc-4ee6-94ab-a98c7231a332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.712126 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b001bac5-34cc-4ee6-94ab-a98c7231a332 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:05.713479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b001bac5-34cc-4ee6-94ab-a98c7231a332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:05.717292 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1365/2714] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:05.728201 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb632a-f1b2-47f7-b8aa-aa149edc16ea None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:05.730598 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b5fb632a-f1b2-47f7-b8aa-aa149edc16ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:05.760615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:05.806250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1350/2715] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:05.820534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb632a-f1b2-47f7-b8aa-aa149edc16ea None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:05.821505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001bac5-34cc-4ee6-94ab-a98c7231a332 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:05.838104 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:05.924930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001bac5-34cc-4ee6-94ab-a98c7231a332 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:05.929564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2dfc5-74ae-4da5-9bfa-cf2ebfbe3f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:05.930228 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2dfc5-74ae-4da5-9bfa-cf2ebfbe3f57 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.930547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2dfc5-74ae-4da5-9bfa-cf2ebfbe3f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.930830 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2dfc5-74ae-4da5-9bfa-cf2ebfbe3f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.935621 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8b271-e1e0-4e6a-bd5e-50d0806b458b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:05.936483 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8b271-e1e0-4e6a-bd5e-50d0806b458b service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:05.936878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8b271-e1e0-4e6a-bd5e-50d0806b458b service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:05.937170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8b271-e1e0-4e6a-bd5e-50d0806b458b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:05.942122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2dfc5-74ae-4da5-9bfa-cf2ebfbe3f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:05.943626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2dfc5-74ae-4da5-9bfa-cf2ebfbe3f57 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:05.950098 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8b271-e1e0-4e6a-bd5e-50d0806b458b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:05.951585 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b8b271-e1e0-4e6a-bd5e-50d0806b458b service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:05.953848 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1351/2716] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:12:05.962144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1366/2717] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:06.409849 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2dfc5-74ae-4da5-9bfa-cf2ebfbe3f57 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:06.412093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1df2dfc5-74ae-4da5-9bfa-cf2ebfbe3f57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:06.416601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8b271-e1e0-4e6a-bd5e-50d0806b458b service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:06.418259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-49b8b271-e1e0-4e6a-bd5e-50d0806b458b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:06.425375 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2dfc5-74ae-4da5-9bfa-cf2ebfbe3f57 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:06.434555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8b271-e1e0-4e6a-bd5e-50d0806b458b service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:06.440149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:06.441029 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:06.441352 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:06.441635 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] PATH_INFO: `/v3/projects/cc72aedcd1094d33a4c1154437aeda34` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:06.443097 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc72aedcd1094d33a4c1154437aeda34)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:06.444589 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:06.445062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:06.445462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:06.445748 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:06.446068 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] PATH_INFO: `/v3/projects/66e8a58b94214f3ca8bb11f97a39785c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:06.452859 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66e8a58b94214f3ca8bb11f97a39785c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:06.454994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:06.498581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc72aedcd1094d33a4c1154437aeda34', 'request_id': 'req-dfd702c9-f579-4d49-868a-6ead97b3e0c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:06.502943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66e8a58b94214f3ca8bb11f97a39785c', 'request_id': 'req-27a240a6-2baf-4df9-b2c9-0c33d4087a62'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:06.505021 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc72aedcd1094d33a4c1154437aeda34', 'request_id': 'req-dfd702c9-f579-4d49-868a-6ead97b3e0c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:06.508493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66e8a58b94214f3ca8bb11f97a39785c', 'request_id': 'req-27a240a6-2baf-4df9-b2c9-0c33d4087a62'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:06.511294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc72aedcd1094d33a4c1154437aeda34', 'request_id': 'req-dfd702c9-f579-4d49-868a-6ead97b3e0c9'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:06.514566 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66e8a58b94214f3ca8bb11f97a39785c', 'request_id': 'req-27a240a6-2baf-4df9-b2c9-0c33d4087a62'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:06.525987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] The token cache is being invalidate because project cc72aedcd1094d33a4c1154437aeda34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:12:06.526283 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:06.526959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-dfd702c9-f579-4d49-868a-6ead97b3e0c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:06.527408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] The token cache is being invalidate because project 66e8a58b94214f3ca8bb11f97a39785c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:12:06.527796 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:06.528519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1367/2718] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:12:06 2022] DELETE /identity/v3/projects/cc72aedcd1094d33a4c1154437aeda34 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:12:06.531863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-27a240a6-2baf-4df9-b2c9-0c33d4087a62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:06.531863 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1352/2719] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:12:06 2022] DELETE /identity/v3/projects/66e8a58b94214f3ca8bb11f97a39785c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:06.533213 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa10066-bc4c-477f-9e03-bb54e57db4bc None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:06.533535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa10066-bc4c-477f-9e03-bb54e57db4bc None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:06.533818 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa10066-bc4c-477f-9e03-bb54e57db4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:06.554740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6fa10066-bc4c-477f-9e03-bb54e57db4bc None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:06.556001 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6fa10066-bc4c-477f-9e03-bb54e57db4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:06.614355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:06.627639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1368/2720] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:06.636539 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b3c6a-ed6c-462f-8a38-5b40dbb15bc3 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:06.636982 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b3c6a-ed6c-462f-8a38-5b40dbb15bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:06.637410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b3c6a-ed6c-462f-8a38-5b40dbb15bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:06.661708 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-dc5b3c6a-ed6c-462f-8a38-5b40dbb15bc3 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:06.662958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-dc5b3c6a-ed6c-462f-8a38-5b40dbb15bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:06.671561 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1353/2721] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:06.675875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa10066-bc4c-477f-9e03-bb54e57db4bc None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:06.688560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa10066-bc4c-477f-9e03-bb54e57db4bc None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:06.695291 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483d504-e96c-4d4c-9ad5-d82c1258eee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:06.695984 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483d504-e96c-4d4c-9ad5-d82c1258eee7 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:06.696269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483d504-e96c-4d4c-9ad5-d82c1258eee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:06.696543 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5483d504-e96c-4d4c-9ad5-d82c1258eee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:06.697286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483d504-e96c-4d4c-9ad5-d82c1258eee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:12:06.697597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483d504-e96c-4d4c-9ad5-d82c1258eee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:06.699440 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5483d504-e96c-4d4c-9ad5-d82c1258eee7 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:06.713890 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1369/2722] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:12:06.718579 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b3c6a-ed6c-462f-8a38-5b40dbb15bc3 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:06.728847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b3c6a-ed6c-462f-8a38-5b40dbb15bc3 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:06.734211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83999721-cd2a-4ffd-abdc-7f24e20af0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:06.735067 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83999721-cd2a-4ffd-abdc-7f24e20af0c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:06.735350 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83999721-cd2a-4ffd-abdc-7f24e20af0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:06.735628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83999721-cd2a-4ffd-abdc-7f24e20af0c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:06.736926 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83999721-cd2a-4ffd-abdc-7f24e20af0c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:06.738721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83999721-cd2a-4ffd-abdc-7f24e20af0c2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:06.773384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1354/2723] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:12:06 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:06.777727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483d504-e96c-4d4c-9ad5-d82c1258eee7 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:06.788534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5483d504-e96c-4d4c-9ad5-d82c1258eee7 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:06.845771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a19405-d521-40e2-b20a-3ecda5e8c099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:06.846438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19405-d521-40e2-b20a-3ecda5e8c099 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:06.846754 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19405-d521-40e2-b20a-3ecda5e8c099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:06.847027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a19405-d521-40e2-b20a-3ecda5e8c099 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:06.847782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a19405-d521-40e2-b20a-3ecda5e8c099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:06.849690 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a19405-d521-40e2-b20a-3ecda5e8c099 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:06.869217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1370/2724] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:12:06 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:06.873247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83999721-cd2a-4ffd-abdc-7f24e20af0c2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:06.884043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83999721-cd2a-4ffd-abdc-7f24e20af0c2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:06.888595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66874a-1b82-4735-9c25-1d0138727aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:06.889247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66874a-1b82-4735-9c25-1d0138727aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:06.889532 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66874a-1b82-4735-9c25-1d0138727aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:06.889813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66874a-1b82-4735-9c25-1d0138727aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:06.890559 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66874a-1b82-4735-9c25-1d0138727aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:06.892171 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66874a-1b82-4735-9c25-1d0138727aa5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:06.904210 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1355/2725] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:06.907685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a19405-d521-40e2-b20a-3ecda5e8c099 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:06.917711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a19405-d521-40e2-b20a-3ecda5e8c099 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:06.922289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31252e85-2c9e-4ebf-9198-767819bd9826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:06.922968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31252e85-2c9e-4ebf-9198-767819bd9826 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:06.923248 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31252e85-2c9e-4ebf-9198-767819bd9826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:06.923522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31252e85-2c9e-4ebf-9198-767819bd9826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:06.924240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31252e85-2c9e-4ebf-9198-767819bd9826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:06.925811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31252e85-2c9e-4ebf-9198-767819bd9826 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:06.935913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1371/2726] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:06.940346 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66874a-1b82-4735-9c25-1d0138727aa5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:06.955329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66874a-1b82-4735-9c25-1d0138727aa5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:06.961005 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5839f1-0583-4b03-a7c2-232473cd3c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:06.961893 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5839f1-0583-4b03-a7c2-232473cd3c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:06.962388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5839f1-0583-4b03-a7c2-232473cd3c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:06.962794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5839f1-0583-4b03-a7c2-232473cd3c0b admin admin] PATH_INFO: `/v3/projects/e74c60fbc90c4993b707cfdbd1051804/users/94eff60bdecb45a3823f99a16a1bf898/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:06.978412 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5839f1-0583-4b03-a7c2-232473cd3c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74c60fbc90c4993b707cfdbd1051804, user_id=94eff60bdecb45a3823f99a16a1bf898, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:06.981144 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5839f1-0583-4b03-a7c2-232473cd3c0b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:06.994614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1356/2727] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:12:06 2022] PUT /identity/v3/projects/e74c60fbc90c4993b707cfdbd1051804/users/94eff60bdecb45a3823f99a16a1bf898/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:07.008555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371afa74-9368-49aa-98e0-be7a71e75063 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:07.008923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371afa74-9368-49aa-98e0-be7a71e75063 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:07.009222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371afa74-9368-49aa-98e0-be7a71e75063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:07.035062 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-371afa74-9368-49aa-98e0-be7a71e75063 None None] MFA Rules not processed for user `94eff60bdecb45a3823f99a16a1bf898`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:07.036259 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-371afa74-9368-49aa-98e0-be7a71e75063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:07.107983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1372/2728] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:07.139130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5839f1-0583-4b03-a7c2-232473cd3c0b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:07.140878 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ea5839f1-0583-4b03-a7c2-232473cd3c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:07.175078 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:07.200304 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5839f1-0583-4b03-a7c2-232473cd3c0b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:07.203996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecdfbf4-07e5-46b7-b478-d933064b0727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:07.204637 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdfbf4-07e5-46b7-b478-d933064b0727 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:07.204924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdfbf4-07e5-46b7-b478-d933064b0727 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:07.205207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdfbf4-07e5-46b7-b478-d933064b0727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:07.213505 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdfbf4-07e5-46b7-b478-d933064b0727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:07.215061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecdfbf4-07e5-46b7-b478-d933064b0727 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:07.224082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1357/2729] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:07.239272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371afa74-9368-49aa-98e0-be7a71e75063 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:07.291660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:07.304528 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371afa74-9368-49aa-98e0-be7a71e75063 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:07.309092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:07.309952 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:07.310232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:07.310538 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] PATH_INFO: `/v3/users/94eff60bdecb45a3823f99a16a1bf898` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:07.311723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94eff60bdecb45a3823f99a16a1bf898)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:07.313641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:07.358283 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94eff60bdecb45a3823f99a16a1bf898'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:07.370959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94eff60bdecb45a3823f99a16a1bf898'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:07.378066 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94eff60bdecb45a3823f99a16a1bf898'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:07.378749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94eff60bdecb45a3823f99a16a1bf898'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:07.385698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94eff60bdecb45a3823f99a16a1bf898'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:07.391649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1373/2730] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:12:07 2022] DELETE /identity/v3/users/94eff60bdecb45a3823f99a16a1bf898 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:07.396156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a5646-c918-452f-87dc-014aaaf1ca39 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:07.396445 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a5646-c918-452f-87dc-014aaaf1ca39 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:07.396727 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a5646-c918-452f-87dc-014aaaf1ca39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:07.413818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-c90a5646-c918-452f-87dc-014aaaf1ca39 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:07.415033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c90a5646-c918-452f-87dc-014aaaf1ca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:07.471562 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1358/2731] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:07.481581 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:07.482998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:07.539385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f20101-46f6-4cd5-84fb-32218a32a920 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:07.542966 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1d5e41-19a0-49b7-a562-627475306dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:07.543600 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d5e41-19a0-49b7-a562-627475306dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:07.543888 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d5e41-19a0-49b7-a562-627475306dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:07.544162 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d5e41-19a0-49b7-a562-627475306dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:07.553730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d5e41-19a0-49b7-a562-627475306dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:07.555183 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1d5e41-19a0-49b7-a562-627475306dd1 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:07.564583 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1374/2732] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:07.880442 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a5646-c918-452f-87dc-014aaaf1ca39 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:07.882675 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-c90a5646-c918-452f-87dc-014aaaf1ca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:07.904674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a5646-c918-452f-87dc-014aaaf1ca39 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:07.911013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:07.912159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:07.912525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:07.912924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] PATH_INFO: `/v3/projects/e74c60fbc90c4993b707cfdbd1051804` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:07.914334 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e74c60fbc90c4993b707cfdbd1051804)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:07.916405 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:07.988180 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e74c60fbc90c4993b707cfdbd1051804', 'request_id': 'req-19561815-4b4f-413a-9409-2bf3eed415bb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:07.995580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e74c60fbc90c4993b707cfdbd1051804', 'request_id': 'req-19561815-4b4f-413a-9409-2bf3eed415bb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.008230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e74c60fbc90c4993b707cfdbd1051804', 'request_id': 'req-19561815-4b4f-413a-9409-2bf3eed415bb'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.023583 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] The token cache is being invalidate because project e74c60fbc90c4993b707cfdbd1051804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:12:08.024032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.024578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-19561815-4b4f-413a-9409-2bf3eed415bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.026230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1359/2733] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:12:07 2022] DELETE /identity/v3/projects/e74c60fbc90c4993b707cfdbd1051804 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:08.031027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b79b85-95b2-4703-93df-932034f35229 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.031507 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b79b85-95b2-4703-93df-932034f35229 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.031615 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b79b85-95b2-4703-93df-932034f35229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.050229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f6b79b85-95b2-4703-93df-932034f35229 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:08.051514 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f6b79b85-95b2-4703-93df-932034f35229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:08.112205 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1375/2734] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:08.120047 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ed7e9-5770-4656-9abf-c54ce45c899c None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.120369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ed7e9-5770-4656-9abf-c54ce45c899c None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.120678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ed7e9-5770-4656-9abf-c54ce45c899c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.135771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b79b85-95b2-4703-93df-932034f35229 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.137158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f6b79b85-95b2-4703-93df-932034f35229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:08.140031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-f26ed7e9-5770-4656-9abf-c54ce45c899c None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:08.141191 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f26ed7e9-5770-4656-9abf-c54ce45c899c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:08.150045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1360/2735] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:12:08.150433 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b79b85-95b2-4703-93df-932034f35229 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:08.154006 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ed7e9-5770-4656-9abf-c54ce45c899c None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.157225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:08.158996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.158996 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.159401 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] PATH_INFO: `/v3/users/27d3c22c7f3b482d9e47910d7bef2eba` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.161427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d3c22c7f3b482d9e47910d7bef2eba)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:08.163731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ed7e9-5770-4656-9abf-c54ce45c899c None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:08.164705 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:08.168779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429188da-5e73-4944-87d3-319b7b0022f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:08.169433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429188da-5e73-4944-87d3-319b7b0022f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.170122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429188da-5e73-4944-87d3-319b7b0022f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.170122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429188da-5e73-4944-87d3-319b7b0022f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.170754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429188da-5e73-4944-87d3-319b7b0022f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:12:08.171066 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429188da-5e73-4944-87d3-319b7b0022f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:08.172866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429188da-5e73-4944-87d3-319b7b0022f0 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:08.181409 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:08.188511 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1361/2736] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:12:08.192201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429188da-5e73-4944-87d3-319b7b0022f0 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.204907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429188da-5e73-4944-87d3-319b7b0022f0 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:08.210983 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa5c2b-37f0-4b70-9416-3fda8e7316a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:08.211673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa5c2b-37f0-4b70-9416-3fda8e7316a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.212116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa5c2b-37f0-4b70-9416-3fda8e7316a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.212414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa5c2b-37f0-4b70-9416-3fda8e7316a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.213117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa5c2b-37f0-4b70-9416-3fda8e7316a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:08.215084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa5c2b-37f0-4b70-9416-3fda8e7316a3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:08.225234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d3c22c7f3b482d9e47910d7bef2eba'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.238082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d3c22c7f3b482d9e47910d7bef2eba'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.241311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1362/2737] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:12:08 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:12:08.245541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa5c2b-37f0-4b70-9416-3fda8e7316a3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.249943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d3c22c7f3b482d9e47910d7bef2eba'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.250767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d3c22c7f3b482d9e47910d7bef2eba'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.258757 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-69c2856a-1490-4ea3-81a8-582ff8fbe2dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d3c22c7f3b482d9e47910d7bef2eba'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.264745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1376/2738] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:12:08 2022] DELETE /identity/v3/users/27d3c22c7f3b482d9e47910d7bef2eba => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:08.265963 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:08.270483 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb602a6-2b05-40b8-9b90-e4f63bfae78b None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.270786 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb602a6-2b05-40b8-9b90-e4f63bfae78b None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.271074 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb602a6-2b05-40b8-9b90-e4f63bfae78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.288879 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-7bb602a6-2b05-40b8-9b90-e4f63bfae78b None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:08.290109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7bb602a6-2b05-40b8-9b90-e4f63bfae78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:08.313128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa5c2b-37f0-4b70-9416-3fda8e7316a3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:08.354299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1377/2739] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:08.365327 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb602a6-2b05-40b8-9b90-e4f63bfae78b None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.366830 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-7bb602a6-2b05-40b8-9b90-e4f63bfae78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:08.376329 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd338e-4490-42b2-97c3-b3bec05c22c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:08.376937 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd338e-4490-42b2-97c3-b3bec05c22c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.377219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd338e-4490-42b2-97c3-b3bec05c22c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.377886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd338e-4490-42b2-97c3-b3bec05c22c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.378183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd338e-4490-42b2-97c3-b3bec05c22c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:08.380677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd338e-4490-42b2-97c3-b3bec05c22c7 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:08.401641 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1363/2740] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:12:08 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:08.406016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd338e-4490-42b2-97c3-b3bec05c22c7 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.407667 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-f1cd338e-4490-42b2-97c3-b3bec05c22c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:08.419792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd338e-4490-42b2-97c3-b3bec05c22c7 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:08.424439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42058bc-49ff-4060-883c-af39da421243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:08.425081 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42058bc-49ff-4060-883c-af39da421243 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.425361 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42058bc-49ff-4060-883c-af39da421243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.425640 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42058bc-49ff-4060-883c-af39da421243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.426348 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42058bc-49ff-4060-883c-af39da421243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:08.426495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb602a6-2b05-40b8-9b90-e4f63bfae78b None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:08.427984 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42058bc-49ff-4060-883c-af39da421243 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:08.430123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7479cb-1fce-4568-884f-f5b6f477707b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:08.430809 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7479cb-1fce-4568-884f-f5b6f477707b service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.431100 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7479cb-1fce-4568-884f-f5b6f477707b service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.431375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7479cb-1fce-4568-884f-f5b6f477707b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.439940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1364/2741] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:08.442110 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42058bc-49ff-4060-883c-af39da421243 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.443628 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7479cb-1fce-4568-884f-f5b6f477707b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:08.445065 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7479cb-1fce-4568-884f-f5b6f477707b service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:08.451981 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42058bc-49ff-4060-883c-af39da421243 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:08.454713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1378/2742] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:12:08.460546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9eac76-20a3-4bba-94a2-388dae1d1f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:08.461288 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eac76-20a3-4bba-94a2-388dae1d1f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.461570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eac76-20a3-4bba-94a2-388dae1d1f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.461844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9eac76-20a3-4bba-94a2-388dae1d1f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.462621 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9eac76-20a3-4bba-94a2-388dae1d1f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:08.464248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9eac76-20a3-4bba-94a2-388dae1d1f30 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:08.475131 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1365/2743] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:12:08.479837 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7479cb-1fce-4568-884f-f5b6f477707b service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.494584 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7479cb-1fce-4568-884f-f5b6f477707b service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:08.500290 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf460c-4121-4f68-8b70-e617ff4e0608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:08.501322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf460c-4121-4f68-8b70-e617ff4e0608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.501696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf460c-4121-4f68-8b70-e617ff4e0608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.502063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf460c-4121-4f68-8b70-e617ff4e0608 admin admin] PATH_INFO: `/v3/projects/9be5606dfadd4b858f13399ddef01f0c/users/316e7211414b4587a8cce7b94311ddd2/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.519731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf460c-4121-4f68-8b70-e617ff4e0608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be5606dfadd4b858f13399ddef01f0c, user_id=316e7211414b4587a8cce7b94311ddd2, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:08.523019 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaf460c-4121-4f68-8b70-e617ff4e0608 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:08.537811 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1379/2744] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:12:08 2022] PUT /identity/v3/projects/9be5606dfadd4b858f13399ddef01f0c/users/316e7211414b4587a8cce7b94311ddd2/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:08.548060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbe91e-09a1-421e-a6f7-bc8defdd4220 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.548433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbe91e-09a1-421e-a6f7-bc8defdd4220 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.548754 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbe91e-09a1-421e-a6f7-bc8defdd4220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.577331 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-09cbe91e-09a1-421e-a6f7-bc8defdd4220 None None] MFA Rules not processed for user `316e7211414b4587a8cce7b94311ddd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:08.578613 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-09cbe91e-09a1-421e-a6f7-bc8defdd4220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:08.657229 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1366/2745] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:12:08.668823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf460c-4121-4f68-8b70-e617ff4e0608 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.670774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-aaaf460c-4121-4f68-8b70-e617ff4e0608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:08.681560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:08.745541 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaf460c-4121-4f68-8b70-e617ff4e0608 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:08.751536 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de6d58-28ea-455a-aa0c-7356e289db60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:08.752343 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de6d58-28ea-455a-aa0c-7356e289db60 service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.752665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de6d58-28ea-455a-aa0c-7356e289db60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.752941 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de6d58-28ea-455a-aa0c-7356e289db60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.763128 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbe91e-09a1-421e-a6f7-bc8defdd4220 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.764763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-09cbe91e-09a1-421e-a6f7-bc8defdd4220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:08.775363 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de6d58-28ea-455a-aa0c-7356e289db60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:08.777493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de6d58-28ea-455a-aa0c-7356e289db60 service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:08.797199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:08.799079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1380/2746] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:08.856681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbe91e-09a1-421e-a6f7-bc8defdd4220 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:08.861636 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:08.862627 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:08.863030 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:08.863384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] PATH_INFO: `/v3/projects/8e3b2848096f437fada2e0c8a0d09e0e` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:08.864938 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e3b2848096f437fada2e0c8a0d09e0e)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:08.866987 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:08.919743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e3b2848096f437fada2e0c8a0d09e0e', 'request_id': 'req-86e34a24-51c3-4873-985c-5b81ca1ccbcf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.926845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e3b2848096f437fada2e0c8a0d09e0e', 'request_id': 'req-86e34a24-51c3-4873-985c-5b81ca1ccbcf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.933513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e3b2848096f437fada2e0c8a0d09e0e', 'request_id': 'req-86e34a24-51c3-4873-985c-5b81ca1ccbcf'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.950694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] The token cache is being invalidate because project 8e3b2848096f437fada2e0c8a0d09e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:12:08.951116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.951881 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:08.954547 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1367/2747] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:12:08 2022] DELETE /identity/v3/projects/8e3b2848096f437fada2e0c8a0d09e0e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:12:08.961109 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de6d58-28ea-455a-aa0c-7356e289db60 service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:08.979168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de6d58-28ea-455a-aa0c-7356e289db60 service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:09.080551 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:09.082714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:09.083186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:09.083465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] PATH_INFO: `/v3/users/0611753741f7490982b73865a33c655f` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:09.084853 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0611753741f7490982b73865a33c655f)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:09.086894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:09.157355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0611753741f7490982b73865a33c655f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:09.174296 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0611753741f7490982b73865a33c655f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:09.183118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0611753741f7490982b73865a33c655f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:09.183867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0611753741f7490982b73865a33c655f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:09.192522 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-28930192-8ff9-4b82-9dab-c6f865dc471c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0611753741f7490982b73865a33c655f'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:09.199503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1381/2748] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:12:08 2022] DELETE /identity/v3/users/0611753741f7490982b73865a33c655f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:09.465360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:09.466861 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:09.480778 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e34a24-51c3-4873-985c-5b81ca1ccbcf admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:09.538570 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:09.539522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:09.539854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:09.540136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] PATH_INFO: `/v3/projects/f4c635fe56ef49e9be1ba44f6c90c858` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:09.541522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c635fe56ef49e9be1ba44f6c90c858)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:09.543491 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:09.589007 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c635fe56ef49e9be1ba44f6c90c858', 'request_id': 'req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:09.593092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:09.595895 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c635fe56ef49e9be1ba44f6c90c858', 'request_id': 'req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:09.603368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c635fe56ef49e9be1ba44f6c90c858', 'request_id': 'req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:09.615854 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] The token cache is being invalidate because project f4c635fe56ef49e9be1ba44f6c90c858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:12:09.616136 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:09.616740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-9f9d5fcf-bd8f-4743-bd97-814b3cbeae1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:09.618262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1368/2749] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:12:09 2022] DELETE /identity/v3/projects/f4c635fe56ef49e9be1ba44f6c90c858 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:09.623547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f765be-69a4-4c83-a66c-f92fd87f4042 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:09.623842 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f765be-69a4-4c83-a66c-f92fd87f4042 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:09.624123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f765be-69a4-4c83-a66c-f92fd87f4042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:09.641370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-c0f765be-69a4-4c83-a66c-f92fd87f4042 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:09.642545 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c0f765be-69a4-4c83-a66c-f92fd87f4042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:09.667301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:09.703758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1382/2750] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:12:09.711372 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eecbd7-8168-4dc9-9631-62741c446267 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:09.711661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eecbd7-8168-4dc9-9631-62741c446267 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:09.711940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eecbd7-8168-4dc9-9631-62741c446267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:09.730170 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-94eecbd7-8168-4dc9-9631-62741c446267 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:09.731392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-94eecbd7-8168-4dc9-9631-62741c446267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:09.740003 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1369/2751] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:09.744161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f765be-69a4-4c83-a66c-f92fd87f4042 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:09.753745 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f765be-69a4-4c83-a66c-f92fd87f4042 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:09.758301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc3e75-8e2b-447d-aa84-29a914bd6929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:09.758982 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc3e75-8e2b-447d-aa84-29a914bd6929 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:09.759268 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc3e75-8e2b-447d-aa84-29a914bd6929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:09.759547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc3e75-8e2b-447d-aa84-29a914bd6929 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:09.760264 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc3e75-8e2b-447d-aa84-29a914bd6929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:12:09.760573 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc3e75-8e2b-447d-aa84-29a914bd6929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:09.762164 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc3e75-8e2b-447d-aa84-29a914bd6929 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:09.774188 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1383/2752] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:12:09.778165 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eecbd7-8168-4dc9-9631-62741c446267 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:09.787990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eecbd7-8168-4dc9-9631-62741c446267 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:09.792607 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb6a0-a180-4cf3-8fc0-f5f06e78cfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:09.793254 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb6a0-a180-4cf3-8fc0-f5f06e78cfa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:09.793536 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb6a0-a180-4cf3-8fc0-f5f06e78cfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:09.793810 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb6a0-a180-4cf3-8fc0-f5f06e78cfa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:09.794522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6eb6a0-a180-4cf3-8fc0-f5f06e78cfa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:09.796429 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6eb6a0-a180-4cf3-8fc0-f5f06e78cfa2 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:09.826275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1370/2753] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:12:09 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:12:09.827462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc3e75-8e2b-447d-aa84-29a914bd6929 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:09.837437 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc3e75-8e2b-447d-aa84-29a914bd6929 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:09.892416 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb00e8-121c-4ac3-b2cb-62ab7ddb542c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:09.892987 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb00e8-121c-4ac3-b2cb-62ab7ddb542c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:09.893269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb00e8-121c-4ac3-b2cb-62ab7ddb542c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:09.893550 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb00e8-121c-4ac3-b2cb-62ab7ddb542c admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:09.894284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb00e8-121c-4ac3-b2cb-62ab7ddb542c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:09.896212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb00e8-121c-4ac3-b2cb-62ab7ddb542c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:09.915681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1384/2754] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:12:09 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:09.919674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb6a0-a180-4cf3-8fc0-f5f06e78cfa2 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:09.930333 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb6a0-a180-4cf3-8fc0-f5f06e78cfa2 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:09.934956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0864b6-a119-48a5-afdb-0b59c6ca5644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:09.935606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0864b6-a119-48a5-afdb-0b59c6ca5644 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:09.935897 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0864b6-a119-48a5-afdb-0b59c6ca5644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:09.936179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0864b6-a119-48a5-afdb-0b59c6ca5644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:09.936903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0864b6-a119-48a5-afdb-0b59c6ca5644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:09.938546 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0864b6-a119-48a5-afdb-0b59c6ca5644 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:09.948661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1371/2755] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:09.952215 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb00e8-121c-4ac3-b2cb-62ab7ddb542c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:09.961895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb00e8-121c-4ac3-b2cb-62ab7ddb542c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:09.966489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105663c-3027-4666-b9d2-ca3c3d0b4e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:09.967137 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105663c-3027-4666-b9d2-ca3c3d0b4e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:09.967424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105663c-3027-4666-b9d2-ca3c3d0b4e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:09.967702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105663c-3027-4666-b9d2-ca3c3d0b4e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:09.968488 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105663c-3027-4666-b9d2-ca3c3d0b4e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:09.970234 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105663c-3027-4666-b9d2-ca3c3d0b4e12 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:09.980805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1385/2756] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:12:09.984744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0864b6-a119-48a5-afdb-0b59c6ca5644 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:09.995384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0864b6-a119-48a5-afdb-0b59c6ca5644 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:09.999907 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6eff02-82ee-4ed0-a4ad-ed8094bf193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:10.000770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6eff02-82ee-4ed0-a4ad-ed8094bf193d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:10.001051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6eff02-82ee-4ed0-a4ad-ed8094bf193d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:10.001326 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6eff02-82ee-4ed0-a4ad-ed8094bf193d admin admin] PATH_INFO: `/v3/projects/49ef00afc24c4954912575a5f7ce2716/users/f719b9d9237941fd9c3c69b0025bffb0/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:10.015871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6eff02-82ee-4ed0-a4ad-ed8094bf193d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49ef00afc24c4954912575a5f7ce2716, user_id=f719b9d9237941fd9c3c69b0025bffb0, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:10.018650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6eff02-82ee-4ed0-a4ad-ed8094bf193d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:10.031520 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1372/2757] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:12:09 2022] PUT /identity/v3/projects/49ef00afc24c4954912575a5f7ce2716/users/f719b9d9237941fd9c3c69b0025bffb0/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:12:10.042085 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ae6a5-7e09-4467-8d16-0fc36379039f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:10.042379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ae6a5-7e09-4467-8d16-0fc36379039f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:10.042714 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ae6a5-7e09-4467-8d16-0fc36379039f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:10.068492 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e79ae6a5-7e09-4467-8d16-0fc36379039f None None] MFA Rules not processed for user `f719b9d9237941fd9c3c69b0025bffb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:10.069824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e79ae6a5-7e09-4467-8d16-0fc36379039f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:10.132048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1386/2758] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:12:10.143488 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6eff02-82ee-4ed0-a4ad-ed8094bf193d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:10.145687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1e6eff02-82ee-4ed0-a4ad-ed8094bf193d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:10.215043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6eff02-82ee-4ed0-a4ad-ed8094bf193d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:10.220243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976c016-c8ee-4414-9075-c60b76401e45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:10.221366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976c016-c8ee-4414-9075-c60b76401e45 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:10.221845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976c016-c8ee-4414-9075-c60b76401e45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:10.222431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976c016-c8ee-4414-9075-c60b76401e45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:10.235456 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976c016-c8ee-4414-9075-c60b76401e45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:10.236966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976c016-c8ee-4414-9075-c60b76401e45 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:10.248828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1373/2759] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:20.129547 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:20.242909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:DELETE=2,keystone:SELECT=58,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:35.091242 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ae6a5-7e09-4467-8d16-0fc36379039f None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:35.152351 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ae6a5-7e09-4467-8d16-0fc36379039f None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:35.156913 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:35.157806 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:35.158093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:35.158376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] PATH_INFO: `/v3/users/f719b9d9237941fd9c3c69b0025bffb0` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:35.159636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f719b9d9237941fd9c3c69b0025bffb0)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:35.162150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:35.212158 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f719b9d9237941fd9c3c69b0025bffb0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:35.229993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f719b9d9237941fd9c3c69b0025bffb0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:35.240329 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f719b9d9237941fd9c3c69b0025bffb0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:35.241127 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f719b9d9237941fd9c3c69b0025bffb0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:35.252803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f719b9d9237941fd9c3c69b0025bffb0'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:35.259650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1387/2760] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:12:35 2022] DELETE /identity/v3/users/f719b9d9237941fd9c3c69b0025bffb0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:35.265236 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89e0d1-8c52-45f5-b8bb-d49294466f05 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:35.265713 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89e0d1-8c52-45f5-b8bb-d49294466f05 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:35.265839 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89e0d1-8c52-45f5-b8bb-d49294466f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:35.287752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ce89e0d1-8c52-45f5-b8bb-d49294466f05 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:35.289023 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ce89e0d1-8c52-45f5-b8bb-d49294466f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:35.357488 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1374/2761] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:35.366734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:35.368137 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:35.435325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f11f-a3b4-4a52-a184-5950016c07b6 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:35.439094 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58065b8-46cd-4ba9-bbaa-a4a717d3c25c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:35.439749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58065b8-46cd-4ba9-bbaa-a4a717d3c25c service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:35.440040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58065b8-46cd-4ba9-bbaa-a4a717d3c25c service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:35.440324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58065b8-46cd-4ba9-bbaa-a4a717d3c25c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:35.450935 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58065b8-46cd-4ba9-bbaa-a4a717d3c25c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:35.452622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58065b8-46cd-4ba9-bbaa-a4a717d3c25c service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:35.466633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1388/2762] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:35.738230 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89e0d1-8c52-45f5-b8bb-d49294466f05 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:35.739952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ce89e0d1-8c52-45f5-b8bb-d49294466f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:35.756787 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89e0d1-8c52-45f5-b8bb-d49294466f05 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:35.762580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:35.763500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:35.763738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:35.764016 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] PATH_INFO: `/v3/projects/49ef00afc24c4954912575a5f7ce2716` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:35.765257 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49ef00afc24c4954912575a5f7ce2716)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:35.767116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:35.814487 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49ef00afc24c4954912575a5f7ce2716', 'request_id': 'req-73cb36a6-360a-4db1-9276-9048c164c4e6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:35.820544 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49ef00afc24c4954912575a5f7ce2716', 'request_id': 'req-73cb36a6-360a-4db1-9276-9048c164c4e6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:35.826680 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49ef00afc24c4954912575a5f7ce2716', 'request_id': 'req-73cb36a6-360a-4db1-9276-9048c164c4e6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:35.839785 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] The token cache is being invalidate because project 49ef00afc24c4954912575a5f7ce2716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:12:35.840075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:35.840699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-73cb36a6-360a-4db1-9276-9048c164c4e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:35.842247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1375/2763] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:12:35 2022] DELETE /identity/v3/projects/49ef00afc24c4954912575a5f7ce2716 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:35.847146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac5c7c-dcf6-42e2-9bd0-4861e36944a4 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:35.847434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac5c7c-dcf6-42e2-9bd0-4861e36944a4 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:35.847717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac5c7c-dcf6-42e2-9bd0-4861e36944a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:35.867735 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-50ac5c7c-dcf6-42e2-9bd0-4861e36944a4 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:35.869192 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-50ac5c7c-dcf6-42e2-9bd0-4861e36944a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:35.889957 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:35.934087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1389/2764] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:35.941580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73625d-784a-494f-8fef-f756248bffdf None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:35.941877 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73625d-784a-494f-8fef-f756248bffdf None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:35.942709 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73625d-784a-494f-8fef-f756248bffdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:35.963978 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-7d73625d-784a-494f-8fef-f756248bffdf None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:35.965159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7d73625d-784a-494f-8fef-f756248bffdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:35.973169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1376/2765] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:35.977314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac5c7c-dcf6-42e2-9bd0-4861e36944a4 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:35.987186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac5c7c-dcf6-42e2-9bd0-4861e36944a4 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:35.992277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f4d91-562b-4900-ab96-fcf826b76500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:35.992927 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4d91-562b-4900-ab96-fcf826b76500 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:35.993210 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4d91-562b-4900-ab96-fcf826b76500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:35.993485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f4d91-562b-4900-ab96-fcf826b76500 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:35.994271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f4d91-562b-4900-ab96-fcf826b76500 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:12:35.994624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f4d91-562b-4900-ab96-fcf826b76500 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:35.996322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f4d91-562b-4900-ab96-fcf826b76500 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:36.008305 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1390/2766] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:12:36.012674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73625d-784a-494f-8fef-f756248bffdf None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:36.022612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73625d-784a-494f-8fef-f756248bffdf None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:36.027360 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e3c8a-c64b-4908-8223-3837037f816c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:36.028009 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3c8a-c64b-4908-8223-3837037f816c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:36.028292 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3c8a-c64b-4908-8223-3837037f816c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:36.028604 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e3c8a-c64b-4908-8223-3837037f816c admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:36.029294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e3c8a-c64b-4908-8223-3837037f816c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:36.031321 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e3c8a-c64b-4908-8223-3837037f816c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:36.055696 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1377/2767] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:12:36 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:36.059821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f4d91-562b-4900-ab96-fcf826b76500 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:36.070299 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f4d91-562b-4900-ab96-fcf826b76500 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:36.125101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf41b17-5382-44bd-a9ab-940c6236ddbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:36.125720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf41b17-5382-44bd-a9ab-940c6236ddbf admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:36.125969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf41b17-5382-44bd-a9ab-940c6236ddbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:36.126251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf41b17-5382-44bd-a9ab-940c6236ddbf admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:36.126964 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf41b17-5382-44bd-a9ab-940c6236ddbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:36.128878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf41b17-5382-44bd-a9ab-940c6236ddbf admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:36.148379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1391/2768] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:12:36 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:36.152402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e3c8a-c64b-4908-8223-3837037f816c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:36.162545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e3c8a-c64b-4908-8223-3837037f816c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:36.167319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf78c39-7efe-4cb6-877b-59c97ddfe373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:36.167974 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78c39-7efe-4cb6-877b-59c97ddfe373 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:36.168255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78c39-7efe-4cb6-877b-59c97ddfe373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:36.168535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78c39-7efe-4cb6-877b-59c97ddfe373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:36.169255 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf78c39-7efe-4cb6-877b-59c97ddfe373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:36.170914 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf78c39-7efe-4cb6-877b-59c97ddfe373 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:36.182308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1378/2769] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:36.186194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf41b17-5382-44bd-a9ab-940c6236ddbf admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:36.197064 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf41b17-5382-44bd-a9ab-940c6236ddbf admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:36.203075 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7395dc-a9bc-4c77-ad9c-83eb411a557c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:36.204050 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7395dc-a9bc-4c77-ad9c-83eb411a557c admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:36.204488 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7395dc-a9bc-4c77-ad9c-83eb411a557c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:36.204926 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7395dc-a9bc-4c77-ad9c-83eb411a557c admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:36.206144 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7395dc-a9bc-4c77-ad9c-83eb411a557c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:36.209263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7395dc-a9bc-4c77-ad9c-83eb411a557c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:36.225678 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1392/2770] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:36.229927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf78c39-7efe-4cb6-877b-59c97ddfe373 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:36.242777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf78c39-7efe-4cb6-877b-59c97ddfe373 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:36.247817 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429bdea-b956-4b5a-82c1-82dc6c032068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:36.248707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429bdea-b956-4b5a-82c1-82dc6c032068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:36.248998 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429bdea-b956-4b5a-82c1-82dc6c032068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:36.249323 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d429bdea-b956-4b5a-82c1-82dc6c032068 admin admin] PATH_INFO: `/v3/projects/3dca56215a11454ca867e663e3d537fd/users/200930e04c7c466e9c99d8664de1e66e/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:36.265542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429bdea-b956-4b5a-82c1-82dc6c032068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dca56215a11454ca867e663e3d537fd, user_id=200930e04c7c466e9c99d8664de1e66e, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:36.268227 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d429bdea-b956-4b5a-82c1-82dc6c032068 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:36.280944 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1379/2771] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:12:36 2022] PUT /identity/v3/projects/3dca56215a11454ca867e663e3d537fd/users/200930e04c7c466e9c99d8664de1e66e/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:12:36.291448 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b25486-41b8-417f-81aa-6bf0b04a28d7 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:36.291866 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b25486-41b8-417f-81aa-6bf0b04a28d7 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:36.291991 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b25486-41b8-417f-81aa-6bf0b04a28d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:36.321063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b1b25486-41b8-417f-81aa-6bf0b04a28d7 None None] MFA Rules not processed for user `200930e04c7c466e9c99d8664de1e66e`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:36.322311 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b1b25486-41b8-417f-81aa-6bf0b04a28d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:36.390521 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1393/2772] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:12:36.402287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429bdea-b956-4b5a-82c1-82dc6c032068 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:36.403990 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d429bdea-b956-4b5a-82c1-82dc6c032068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:36.428376 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:36.467266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d429bdea-b956-4b5a-82c1-82dc6c032068 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:36.470876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2a80e4-ad1d-4b96-bf84-9e82a74ce4f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:36.471509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a80e4-ad1d-4b96-bf84-9e82a74ce4f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:36.471788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a80e4-ad1d-4b96-bf84-9e82a74ce4f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:36.472061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2a80e4-ad1d-4b96-bf84-9e82a74ce4f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:36.480661 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a80e4-ad1d-4b96-bf84-9e82a74ce4f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:36.482252 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2a80e4-ad1d-4b96-bf84-9e82a74ce4f8 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:36.491997 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1380/2773] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:36.517993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b25486-41b8-417f-81aa-6bf0b04a28d7 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:36.542699 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:36.609608 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b25486-41b8-417f-81aa-6bf0b04a28d7 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:36.616763 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:36.618195 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:36.618670 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:36.619138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] PATH_INFO: `/v3/users/200930e04c7c466e9c99d8664de1e66e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:36.620819 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] RBAC: Authorizing `identity:delete_user(user_id=200930e04c7c466e9c99d8664de1e66e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:36.622869 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:36.673880 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200930e04c7c466e9c99d8664de1e66e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:36.696936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200930e04c7c466e9c99d8664de1e66e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:36.708212 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200930e04c7c466e9c99d8664de1e66e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:36.709332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200930e04c7c466e9c99d8664de1e66e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:36.730301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200930e04c7c466e9c99d8664de1e66e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:36.737135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1394/2774] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:12:36 2022] DELETE /identity/v3/users/200930e04c7c466e9c99d8664de1e66e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:36.742594 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43fdad-cff5-4c39-952c-b0ae8db18064 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:36.742957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43fdad-cff5-4c39-952c-b0ae8db18064 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:36.743408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43fdad-cff5-4c39-952c-b0ae8db18064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:36.761007 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-1b43fdad-cff5-4c39-952c-b0ae8db18064 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:36.762387 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1b43fdad-cff5-4c39-952c-b0ae8db18064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:36.854090 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1381/2775] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:36.864710 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:36.866211 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:36.924240 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231de47-704b-4e7c-b75d-e278758f563a admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:36.928426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401066e7-1c37-460d-b603-1b4abf8a37a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:36.929070 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401066e7-1c37-460d-b603-1b4abf8a37a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:36.929357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401066e7-1c37-460d-b603-1b4abf8a37a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:36.929652 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401066e7-1c37-460d-b603-1b4abf8a37a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:36.938493 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401066e7-1c37-460d-b603-1b4abf8a37a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:36.939923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401066e7-1c37-460d-b603-1b4abf8a37a1 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:36.949204 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1395/2776] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:37.214834 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43fdad-cff5-4c39-952c-b0ae8db18064 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:37.216333 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-1b43fdad-cff5-4c39-952c-b0ae8db18064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:37.234238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43fdad-cff5-4c39-952c-b0ae8db18064 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:37.241572 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:37.242334 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.242669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.242928 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] PATH_INFO: `/v3/projects/3dca56215a11454ca867e663e3d537fd` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.244201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dca56215a11454ca867e663e3d537fd)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:37.248340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:37.299515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dca56215a11454ca867e663e3d537fd', 'request_id': 'req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:37.304899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dca56215a11454ca867e663e3d537fd', 'request_id': 'req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:37.310999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dca56215a11454ca867e663e3d537fd', 'request_id': 'req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:37.325081 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] The token cache is being invalidate because project 3dca56215a11454ca867e663e3d537fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:12:37.325391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:37.326063 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-c17049d0-e47a-4592-b3ad-1b243b5e4a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:12:37.327743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1382/2777] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:12:37 2022] DELETE /identity/v3/projects/3dca56215a11454ca867e663e3d537fd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:37.333376 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1698f-29fb-4526-b6fe-8c524f537a4d None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.333900 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1698f-29fb-4526-b6fe-8c524f537a4d None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.334008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1698f-29fb-4526-b6fe-8c524f537a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.351354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-33b1698f-29fb-4526-b6fe-8c524f537a4d None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:37.352560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-33b1698f-29fb-4526-b6fe-8c524f537a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:37.387633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:37.414287 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1396/2778] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:12:37.421842 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77b8b8-7cf6-406c-9305-981f23d08559 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.422129 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77b8b8-7cf6-406c-9305-981f23d08559 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.422408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77b8b8-7cf6-406c-9305-981f23d08559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.442870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-9d77b8b8-7cf6-406c-9305-981f23d08559 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:37.444102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-9d77b8b8-7cf6-406c-9305-981f23d08559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:37.452053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1383/2779] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:37.456312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1698f-29fb-4526-b6fe-8c524f537a4d None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:37.466898 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1698f-29fb-4526-b6fe-8c524f537a4d None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:37.471622 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44981f-38c6-4655-aaf0-4c1d6f68179d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:37.472288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44981f-38c6-4655-aaf0-4c1d6f68179d admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.472574 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44981f-38c6-4655-aaf0-4c1d6f68179d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.472854 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44981f-38c6-4655-aaf0-4c1d6f68179d admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.473624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44981f-38c6-4655-aaf0-4c1d6f68179d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:12:37.473934 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44981f-38c6-4655-aaf0-4c1d6f68179d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:37.475601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44981f-38c6-4655-aaf0-4c1d6f68179d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:37.487099 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1397/2780] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:12:37.491524 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77b8b8-7cf6-406c-9305-981f23d08559 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:37.500736 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77b8b8-7cf6-406c-9305-981f23d08559 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:37.505866 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b0ab4-220c-4fd9-8711-3929c3764c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:37.506526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b0ab4-220c-4fd9-8711-3929c3764c43 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.506813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b0ab4-220c-4fd9-8711-3929c3764c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.507092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b0ab4-220c-4fd9-8711-3929c3764c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.507788 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b0ab4-220c-4fd9-8711-3929c3764c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:37.509694 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b0ab4-220c-4fd9-8711-3929c3764c43 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:37.537768 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1384/2781] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:12:37 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:37.541611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44981f-38c6-4655-aaf0-4c1d6f68179d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:37.554632 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44981f-38c6-4655-aaf0-4c1d6f68179d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:37.613123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d95a5-ce5e-4490-930e-656e4826a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:37.613814 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d95a5-ce5e-4490-930e-656e4826a30c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.614372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d95a5-ce5e-4490-930e-656e4826a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.614679 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d95a5-ce5e-4490-930e-656e4826a30c admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.615369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d95a5-ce5e-4490-930e-656e4826a30c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:37.617286 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d95a5-ce5e-4490-930e-656e4826a30c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:37.641367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1398/2782] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:12:37 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:12:37.642509 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b0ab4-220c-4fd9-8711-3929c3764c43 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:37.653192 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b0ab4-220c-4fd9-8711-3929c3764c43 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:37.657832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb90888-2a62-464b-854d-1cac42e4aa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:37.658704 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb90888-2a62-464b-854d-1cac42e4aa74 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.659118 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb90888-2a62-464b-854d-1cac42e4aa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.659402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb90888-2a62-464b-854d-1cac42e4aa74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.660122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb90888-2a62-464b-854d-1cac42e4aa74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:37.661752 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb90888-2a62-464b-854d-1cac42e4aa74 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:37.676629 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1385/2783] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:12:37.681289 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d95a5-ce5e-4490-930e-656e4826a30c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:37.691870 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d95a5-ce5e-4490-930e-656e4826a30c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:37.697140 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e5b8c-ec45-467c-98f3-691087f61572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:37.697799 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e5b8c-ec45-467c-98f3-691087f61572 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.698080 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e5b8c-ec45-467c-98f3-691087f61572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.698355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e5b8c-ec45-467c-98f3-691087f61572 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.699106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e5b8c-ec45-467c-98f3-691087f61572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:37.700693 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e5b8c-ec45-467c-98f3-691087f61572 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:37.710997 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1399/2784] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:12:37.715073 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb90888-2a62-464b-854d-1cac42e4aa74 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:37.724875 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb90888-2a62-464b-854d-1cac42e4aa74 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:37.729437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9684fd-b763-4eb0-bd3f-0701cc40853d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:37.730305 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9684fd-b763-4eb0-bd3f-0701cc40853d admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.730629 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9684fd-b763-4eb0-bd3f-0701cc40853d admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.730908 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9684fd-b763-4eb0-bd3f-0701cc40853d admin admin] PATH_INFO: `/v3/projects/991ad84a43c747cf98063b7b284ffe01/users/592569330e4345c59c6c05426d34475e/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.733088 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:37.747366 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9684fd-b763-4eb0-bd3f-0701cc40853d admin admin] RBAC: Authorizing `identity:create_grant(project_id=991ad84a43c747cf98063b7b284ffe01, user_id=592569330e4345c59c6c05426d34475e, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:37.750002 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9684fd-b763-4eb0-bd3f-0701cc40853d admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:37.763308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1386/2785] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:12:37 2022] PUT /identity/v3/projects/991ad84a43c747cf98063b7b284ffe01/users/592569330e4345c59c6c05426d34475e/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:12:37.773148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265e156-68cc-4385-b474-63fc9a517fc2 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.773443 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265e156-68cc-4385-b474-63fc9a517fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.773720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265e156-68cc-4385-b474-63fc9a517fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.809883 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6265e156-68cc-4385-b474-63fc9a517fc2 None None] MFA Rules not processed for user `592569330e4345c59c6c05426d34475e`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:12:37.811333 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6265e156-68cc-4385-b474-63fc9a517fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:37.879314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1400/2786] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:12:37.891529 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9684fd-b763-4eb0-bd3f-0701cc40853d admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:12:37.893357 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-8c9684fd-b763-4eb0-bd3f-0701cc40853d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:12:37.952671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9684fd-b763-4eb0-bd3f-0701cc40853d admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:12:37.956545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cb6e2-5383-4082-a4ee-d7cb74afc851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:12:37.957302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cb6e2-5383-4082-a4ee-d7cb74afc851 service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:12:37.957738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cb6e2-5383-4082-a4ee-d7cb74afc851 service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:12:37.958026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cb6e2-5383-4082-a4ee-d7cb74afc851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:12:37.966916 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cb6e2-5383-4082-a4ee-d7cb74afc851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:12:37.968401 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cb6e2-5383-4082-a4ee-d7cb74afc851 service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:12:37.978330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1387/2787] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:12:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:12:47.876346 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:12:47.974044 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:01.473958 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265e156-68cc-4385-b474-63fc9a517fc2 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:01.545256 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265e156-68cc-4385-b474-63fc9a517fc2 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:01.549755 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:01.550668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:01.550956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:01.551237 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] PATH_INFO: `/v3/users/592569330e4345c59c6c05426d34475e` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:01.552424 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] RBAC: Authorizing `identity:delete_user(user_id=592569330e4345c59c6c05426d34475e)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:01.554332 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:01.598843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592569330e4345c59c6c05426d34475e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:01.611535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592569330e4345c59c6c05426d34475e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:01.620681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592569330e4345c59c6c05426d34475e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:01.621504 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592569330e4345c59c6c05426d34475e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:01.628623 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592569330e4345c59c6c05426d34475e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:01.635063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1401/2788] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:13:01 2022] DELETE /identity/v3/users/592569330e4345c59c6c05426d34475e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:01.639738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327fc0f-46ad-40df-afbf-46af7dc01fa7 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:01.640222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327fc0f-46ad-40df-afbf-46af7dc01fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:01.640343 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327fc0f-46ad-40df-afbf-46af7dc01fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:01.656885 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-6327fc0f-46ad-40df-afbf-46af7dc01fa7 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:01.658061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6327fc0f-46ad-40df-afbf-46af7dc01fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:01.724683 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1388/2789] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:01.732087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:01.733445 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:01.795938 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec1c4e-f707-4f12-b76b-5f53484d80de admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:01.806439 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413db51-1dd5-45e6-9acd-3c0d87817040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:01.807758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413db51-1dd5-45e6-9acd-3c0d87817040 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:01.808048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413db51-1dd5-45e6-9acd-3c0d87817040 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:01.808328 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413db51-1dd5-45e6-9acd-3c0d87817040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:01.819771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413db51-1dd5-45e6-9acd-3c0d87817040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:01.821318 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413db51-1dd5-45e6-9acd-3c0d87817040 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:01.831456 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1402/2790] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:13:02.104025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327fc0f-46ad-40df-afbf-46af7dc01fa7 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:02.106792 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6327fc0f-46ad-40df-afbf-46af7dc01fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:02.122084 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327fc0f-46ad-40df-afbf-46af7dc01fa7 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:02.128674 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:02.130781 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.131079 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.131368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] PATH_INFO: `/v3/projects/991ad84a43c747cf98063b7b284ffe01` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.132617 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=991ad84a43c747cf98063b7b284ffe01)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:02.134614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:02.184336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '991ad84a43c747cf98063b7b284ffe01', 'request_id': 'req-af1defca-dea1-4511-bf48-6ad0d4aabd40'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:02.190947 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '991ad84a43c747cf98063b7b284ffe01', 'request_id': 'req-af1defca-dea1-4511-bf48-6ad0d4aabd40'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:02.197159 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '991ad84a43c747cf98063b7b284ffe01', 'request_id': 'req-af1defca-dea1-4511-bf48-6ad0d4aabd40'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:02.209479 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] The token cache is being invalidate because project 991ad84a43c747cf98063b7b284ffe01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:13:02.209771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:02.210389 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-af1defca-dea1-4511-bf48-6ad0d4aabd40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:02.211966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1389/2791] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:13:02 2022] DELETE /identity/v3/projects/991ad84a43c747cf98063b7b284ffe01 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:02.217381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784ed07-7bc1-44af-9ee0-13fbefcfb790 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.217681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784ed07-7bc1-44af-9ee0-13fbefcfb790 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.217970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784ed07-7bc1-44af-9ee0-13fbefcfb790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.236319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6784ed07-7bc1-44af-9ee0-13fbefcfb790 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:02.237523 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6784ed07-7bc1-44af-9ee0-13fbefcfb790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:02.254718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:02.298659 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1403/2792] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:02.306548 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432392-a68d-4aff-ac04-a3ddb11df7a5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.306851 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432392-a68d-4aff-ac04-a3ddb11df7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.307140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432392-a68d-4aff-ac04-a3ddb11df7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.325891 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-cf432392-a68d-4aff-ac04-a3ddb11df7a5 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:02.327125 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-cf432392-a68d-4aff-ac04-a3ddb11df7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:02.335181 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1390/2793] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:02.339914 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784ed07-7bc1-44af-9ee0-13fbefcfb790 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:02.349415 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784ed07-7bc1-44af-9ee0-13fbefcfb790 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:02.354115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52325c5-fd95-4bbe-8110-a148634bb2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:02.354734 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52325c5-fd95-4bbe-8110-a148634bb2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.355027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52325c5-fd95-4bbe-8110-a148634bb2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.355314 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52325c5-fd95-4bbe-8110-a148634bb2b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.356038 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52325c5-fd95-4bbe-8110-a148634bb2b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:13:02.356354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52325c5-fd95-4bbe-8110-a148634bb2b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:02.357959 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52325c5-fd95-4bbe-8110-a148634bb2b2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:02.369703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1404/2794] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:13:02.373497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432392-a68d-4aff-ac04-a3ddb11df7a5 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:02.382490 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432392-a68d-4aff-ac04-a3ddb11df7a5 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:02.387589 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4820868-2abe-468b-a4bd-59b3f7d0f739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:02.388219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4820868-2abe-468b-a4bd-59b3f7d0f739 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.388500 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4820868-2abe-468b-a4bd-59b3f7d0f739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.388777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4820868-2abe-468b-a4bd-59b3f7d0f739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.389481 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4820868-2abe-468b-a4bd-59b3f7d0f739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:02.391414 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4820868-2abe-468b-a4bd-59b3f7d0f739 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:02.416179 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1391/2795] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:13:02 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:13:02.420255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52325c5-fd95-4bbe-8110-a148634bb2b2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:02.430421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52325c5-fd95-4bbe-8110-a148634bb2b2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:02.486617 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90dee0-6e46-4576-b43b-fb1596f78f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:02.487267 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90dee0-6e46-4576-b43b-fb1596f78f9c admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.487557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90dee0-6e46-4576-b43b-fb1596f78f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.487832 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90dee0-6e46-4576-b43b-fb1596f78f9c admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.488508 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90dee0-6e46-4576-b43b-fb1596f78f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:02.490706 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90dee0-6e46-4576-b43b-fb1596f78f9c admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:02.510513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1405/2796] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:13:02 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:13:02.515075 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4820868-2abe-468b-a4bd-59b3f7d0f739 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:02.525227 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4820868-2abe-468b-a4bd-59b3f7d0f739 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:02.530368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e655a-09ee-492e-8be7-32784835d83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:02.531058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e655a-09ee-492e-8be7-32784835d83f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.531336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e655a-09ee-492e-8be7-32784835d83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.531614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e655a-09ee-492e-8be7-32784835d83f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.532325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e655a-09ee-492e-8be7-32784835d83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:02.533973 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e655a-09ee-492e-8be7-32784835d83f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:02.545069 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1392/2797] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:13:02.549214 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90dee0-6e46-4576-b43b-fb1596f78f9c admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:02.559427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90dee0-6e46-4576-b43b-fb1596f78f9c admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:02.564155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2eb342-2419-4079-af49-834811203148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:02.564817 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb342-2419-4079-af49-834811203148 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.565106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb342-2419-4079-af49-834811203148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.565388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2eb342-2419-4079-af49-834811203148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.566146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2eb342-2419-4079-af49-834811203148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:02.567779 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2eb342-2419-4079-af49-834811203148 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:02.581411 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1406/2798] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:13:02.582954 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e655a-09ee-492e-8be7-32784835d83f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:02.593215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e655a-09ee-492e-8be7-32784835d83f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:02.597808 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0e242-5498-44e7-9f8b-1f382ae0359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:02.598706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e242-5498-44e7-9f8b-1f382ae0359b admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.598992 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e242-5498-44e7-9f8b-1f382ae0359b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.599267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0e242-5498-44e7-9f8b-1f382ae0359b admin admin] PATH_INFO: `/v3/projects/759c767b215942e5852e12fa01e22001/users/5aa8001294eb45179e9b51a8974beb8a/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.615302 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0e242-5498-44e7-9f8b-1f382ae0359b admin admin] RBAC: Authorizing `identity:create_grant(project_id=759c767b215942e5852e12fa01e22001, user_id=5aa8001294eb45179e9b51a8974beb8a, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:02.618334 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0e242-5498-44e7-9f8b-1f382ae0359b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:02.632779 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1393/2799] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:13:02 2022] PUT /identity/v3/projects/759c767b215942e5852e12fa01e22001/users/5aa8001294eb45179e9b51a8974beb8a/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:02.638180 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1730a-ccdf-48a0-a932-003101eadb4a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.638513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1730a-ccdf-48a0-a932-003101eadb4a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.638805 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1730a-ccdf-48a0-a932-003101eadb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.658335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d4a1730a-ccdf-48a0-a932-003101eadb4a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:02.659582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d4a1730a-ccdf-48a0-a932-003101eadb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:02.722921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1407/2800] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:02.732869 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0e242-5498-44e7-9f8b-1f382ae0359b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:02.734620 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5dc0e242-5498-44e7-9f8b-1f382ae0359b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:02.756496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:02.792789 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0e242-5498-44e7-9f8b-1f382ae0359b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:02.796398 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ef427-501d-476a-ae1f-72fa31a11977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:02.797036 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ef427-501d-476a-ae1f-72fa31a11977 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:02.797320 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ef427-501d-476a-ae1f-72fa31a11977 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:02.797598 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ef427-501d-476a-ae1f-72fa31a11977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:02.806892 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ef427-501d-476a-ae1f-72fa31a11977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:02.808864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ef427-501d-476a-ae1f-72fa31a11977 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:02.818699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1394/2801] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:13:04.869405 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1730a-ccdf-48a0-a932-003101eadb4a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:04.871156 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d4a1730a-ccdf-48a0-a932-003101eadb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:04.884133 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1730a-ccdf-48a0-a932-003101eadb4a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:04.890669 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:04.891667 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:04.892027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:04.892375 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] PATH_INFO: `/v3/users/316e7211414b4587a8cce7b94311ddd2` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:04.894272 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=316e7211414b4587a8cce7b94311ddd2)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:04.896319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:04.922800 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:04.960025 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '316e7211414b4587a8cce7b94311ddd2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:04.975269 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '316e7211414b4587a8cce7b94311ddd2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:04.983874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '316e7211414b4587a8cce7b94311ddd2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:04.985428 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '316e7211414b4587a8cce7b94311ddd2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:04.994795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '316e7211414b4587a8cce7b94311ddd2'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:05.001994 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1408/2802] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:13:04 2022] DELETE /identity/v3/users/316e7211414b4587a8cce7b94311ddd2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:05.006599 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c53741-2d0b-4ed0-8bcf-131b30a19490 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:05.007122 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c53741-2d0b-4ed0-8bcf-131b30a19490 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:05.007219 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c53741-2d0b-4ed0-8bcf-131b30a19490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:05.025653 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-42c53741-2d0b-4ed0-8bcf-131b30a19490 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:05.026872 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-42c53741-2d0b-4ed0-8bcf-131b30a19490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:05.122574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1395/2803] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:05.131533 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:05.133309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:05.204082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8954b9b-e405-4b58-8f1d-041f3ced22ad admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:05.209434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c55c1-1196-474b-8f5e-69b74284083d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:05.210284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c55c1-1196-474b-8f5e-69b74284083d service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:05.210767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c55c1-1196-474b-8f5e-69b74284083d service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:05.211115 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c55c1-1196-474b-8f5e-69b74284083d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:05.222833 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c55c1-1196-474b-8f5e-69b74284083d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:05.225979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c55c1-1196-474b-8f5e-69b74284083d service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:05.243221 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1409/2804] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:13:05.605873 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c53741-2d0b-4ed0-8bcf-131b30a19490 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:05.607368 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-42c53741-2d0b-4ed0-8bcf-131b30a19490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:05.624250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c53741-2d0b-4ed0-8bcf-131b30a19490 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:05.630274 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:05.631213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:05.631686 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:05.632184 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] PATH_INFO: `/v3/projects/9be5606dfadd4b858f13399ddef01f0c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:05.633437 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be5606dfadd4b858f13399ddef01f0c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:05.635463 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:05.690943 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be5606dfadd4b858f13399ddef01f0c', 'request_id': 'req-e9a2bab7-5377-41f7-9f83-955c791e7e72'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:05.700644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be5606dfadd4b858f13399ddef01f0c', 'request_id': 'req-e9a2bab7-5377-41f7-9f83-955c791e7e72'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:05.708050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be5606dfadd4b858f13399ddef01f0c', 'request_id': 'req-e9a2bab7-5377-41f7-9f83-955c791e7e72'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:05.722818 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] The token cache is being invalidate because project 9be5606dfadd4b858f13399ddef01f0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:13:05.723032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:05.724248 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-e9a2bab7-5377-41f7-9f83-955c791e7e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:05.725972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1396/2805] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:13:05 2022] DELETE /identity/v3/projects/9be5606dfadd4b858f13399ddef01f0c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:05.731282 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a025b-13c0-4084-87b5-f62f5eb2f224 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:05.731582 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a025b-13c0-4084-87b5-f62f5eb2f224 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:05.731864 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156a025b-13c0-4084-87b5-f62f5eb2f224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:05.751467 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-156a025b-13c0-4084-87b5-f62f5eb2f224 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:05.752697 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-156a025b-13c0-4084-87b5-f62f5eb2f224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:05.828683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1410/2806] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:05.836072 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b047c-a615-40d1-9f34-1b6d8fc0348f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:05.836464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b047c-a615-40d1-9f34-1b6d8fc0348f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:05.836665 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b047c-a615-40d1-9f34-1b6d8fc0348f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:05.859033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ea7b047c-a615-40d1-9f34-1b6d8fc0348f None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:05.860275 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ea7b047c-a615-40d1-9f34-1b6d8fc0348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:05.874268 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1397/2807] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:05.875867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a025b-13c0-4084-87b5-f62f5eb2f224 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:05.887432 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156a025b-13c0-4084-87b5-f62f5eb2f224 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:05.892863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178e27c-585f-40dc-8327-62810f61f0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:05.893720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e27c-585f-40dc-8327-62810f61f0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:05.894312 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e27c-585f-40dc-8327-62810f61f0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:05.894737 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e27c-585f-40dc-8327-62810f61f0e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:05.895899 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178e27c-585f-40dc-8327-62810f61f0e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80268) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:13:05.896520 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178e27c-585f-40dc-8327-62810f61f0e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:05.898245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178e27c-585f-40dc-8327-62810f61f0e3 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:05.915149 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1411/2808] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:13:05.916152 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b047c-a615-40d1-9f34-1b6d8fc0348f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:05.926923 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b047c-a615-40d1-9f34-1b6d8fc0348f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:05.933092 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bce52b-2dfe-4ba0-858e-9129718f6c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:05.933867 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bce52b-2dfe-4ba0-858e-9129718f6c85 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:05.935574 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bce52b-2dfe-4ba0-858e-9129718f6c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:05.935945 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bce52b-2dfe-4ba0-858e-9129718f6c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:05.936729 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bce52b-2dfe-4ba0-858e-9129718f6c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:05.939100 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bce52b-2dfe-4ba0-858e-9129718f6c85 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:05.978169 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1398/2809] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:13:05 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:13:05.979123 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178e27c-585f-40dc-8327-62810f61f0e3 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:05.990534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178e27c-585f-40dc-8327-62810f61f0e3 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:06.037561 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:06.085209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8488ebd-45bc-4aa9-9e85-4e5c66cc3bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:06.087120 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8488ebd-45bc-4aa9-9e85-4e5c66cc3bef admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:06.089151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8488ebd-45bc-4aa9-9e85-4e5c66cc3bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:06.089519 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8488ebd-45bc-4aa9-9e85-4e5c66cc3bef admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:06.090309 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8488ebd-45bc-4aa9-9e85-4e5c66cc3bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:06.092368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8488ebd-45bc-4aa9-9e85-4e5c66cc3bef admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:06.141302 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1412/2810] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:13:05 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:13:06.146145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bce52b-2dfe-4ba0-858e-9129718f6c85 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:06.165994 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bce52b-2dfe-4ba0-858e-9129718f6c85 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:06.173410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433f19-981c-4de6-9728-3f244785fc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:06.174139 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433f19-981c-4de6-9728-3f244785fc2f admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:06.174496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433f19-981c-4de6-9728-3f244785fc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:06.174777 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7433f19-981c-4de6-9728-3f244785fc2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:06.182340 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7433f19-981c-4de6-9728-3f244785fc2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:06.184980 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7433f19-981c-4de6-9728-3f244785fc2f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:06.202770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1399/2811] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:13:06.203401 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8488ebd-45bc-4aa9-9e85-4e5c66cc3bef admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:06.215956 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8488ebd-45bc-4aa9-9e85-4e5c66cc3bef admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:06.221231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1a993-3ed4-4305-a2fc-6b0ea021c21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:06.221911 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1a993-3ed4-4305-a2fc-6b0ea021c21f admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:06.222196 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1a993-3ed4-4305-a2fc-6b0ea021c21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:06.222557 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1a993-3ed4-4305-a2fc-6b0ea021c21f admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:06.223372 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1a993-3ed4-4305-a2fc-6b0ea021c21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:06.224847 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1a993-3ed4-4305-a2fc-6b0ea021c21f admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:06.238020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1413/2812] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:13:06.243108 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433f19-981c-4de6-9728-3f244785fc2f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:06.259020 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7433f19-981c-4de6-9728-3f244785fc2f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:06.266564 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79369cb-4ead-45f9-a390-d795b171a62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:06.268605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79369cb-4ead-45f9-a390-d795b171a62f admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:06.268991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79369cb-4ead-45f9-a390-d795b171a62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:06.269342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79369cb-4ead-45f9-a390-d795b171a62f admin admin] PATH_INFO: `/v3/projects/2ac6e23534754c618cebf75ae3e36eb6/users/6816e6749679401cbfdd6cc8bcf050e5/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:06.272513 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:06.290898 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79369cb-4ead-45f9-a390-d795b171a62f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ac6e23534754c618cebf75ae3e36eb6, user_id=6816e6749679401cbfdd6cc8bcf050e5, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:06.295166 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79369cb-4ead-45f9-a390-d795b171a62f admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:06.312618 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1400/2813] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:13:06 2022] PUT /identity/v3/projects/2ac6e23534754c618cebf75ae3e36eb6/users/6816e6749679401cbfdd6cc8bcf050e5/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:06.322993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b8d89-6d24-4ec1-ab06-c5541c62e62f None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:06.324079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b8d89-6d24-4ec1-ab06-c5541c62e62f None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:06.324368 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b8d89-6d24-4ec1-ab06-c5541c62e62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:06.358431 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-fc2b8d89-6d24-4ec1-ab06-c5541c62e62f None None] MFA Rules not processed for user `6816e6749679401cbfdd6cc8bcf050e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:06.359707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fc2b8d89-6d24-4ec1-ab06-c5541c62e62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:06.454473 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1414/2814] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:06.470294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79369cb-4ead-45f9-a390-d795b171a62f admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:06.472689 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a79369cb-4ead-45f9-a390-d795b171a62f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:06.561742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79369cb-4ead-45f9-a390-d795b171a62f admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:06.566644 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3496ab-b1a3-49db-83d9-999b26207d7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:06.567292 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3496ab-b1a3-49db-83d9-999b26207d7f service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:06.567578 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3496ab-b1a3-49db-83d9-999b26207d7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:06.568146 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3496ab-b1a3-49db-83d9-999b26207d7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:06.578124 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3496ab-b1a3-49db-83d9-999b26207d7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:06.579612 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3496ab-b1a3-49db-83d9-999b26207d7f service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:06.591223 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1401/2815] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:13:07.034277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b241b-e992-4f54-990c-2b0fccb9aca8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:07.035422 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b241b-e992-4f54-990c-2b0fccb9aca8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:07.035884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b241b-e992-4f54-990c-2b0fccb9aca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:07.069186 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-8e2b241b-e992-4f54-990c-2b0fccb9aca8 None None] MFA Rules not processed for user `5aa8001294eb45179e9b51a8974beb8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:07.070534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8e2b241b-e992-4f54-990c-2b0fccb9aca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:07.145636 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1415/2816] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:13:07.150347 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3496ab-b1a3-49db-83d9-999b26207d7f service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:07.152492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7c3496ab-b1a3-49db-83d9-999b26207d7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:07.219062 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3496ab-b1a3-49db-83d9-999b26207d7f service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:07.223533 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3b72f-453a-43fc-932b-5a57f09481a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:07.224194 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3b72f-453a-43fc-932b-5a57f09481a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:07.224483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3b72f-453a-43fc-932b-5a57f09481a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:07.224761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d3b72f-453a-43fc-932b-5a57f09481a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:07.225458 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3b72f-453a-43fc-932b-5a57f09481a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:07.227443 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d3b72f-453a-43fc-932b-5a57f09481a9 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:07.254952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1402/2817] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:13:07 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:13:07.258932 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b241b-e992-4f54-990c-2b0fccb9aca8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:07.283532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b241b-e992-4f54-990c-2b0fccb9aca8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:07.320345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:07.357781 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0b70-417d-4ea9-b0ad-2d6aa5018fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:07.358495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0b70-417d-4ea9-b0ad-2d6aa5018fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:07.358785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0b70-417d-4ea9-b0ad-2d6aa5018fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:07.359059 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0b70-417d-4ea9-b0ad-2d6aa5018fa1 admin admin] PATH_INFO: `/v3/users` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:07.359766 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f0b70-417d-4ea9-b0ad-2d6aa5018fa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:07.361676 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f0b70-417d-4ea9-b0ad-2d6aa5018fa1 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:07.382845 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1416/2818] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:13:07 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:13:07.387394 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3b72f-453a-43fc-932b-5a57f09481a9 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:07.398910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d3b72f-453a-43fc-932b-5a57f09481a9 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:07.403958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be66c5-04c4-4dac-a887-d5546321d39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:07.404766 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be66c5-04c4-4dac-a887-d5546321d39b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:07.405080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be66c5-04c4-4dac-a887-d5546321d39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:07.405556 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be66c5-04c4-4dac-a887-d5546321d39b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:07.406298 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be66c5-04c4-4dac-a887-d5546321d39b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:07.407995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be66c5-04c4-4dac-a887-d5546321d39b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:07.422899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1403/2819] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:13:07.427012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0b70-417d-4ea9-b0ad-2d6aa5018fa1 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:07.438227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0b70-417d-4ea9-b0ad-2d6aa5018fa1 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:07.443319 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e413ca-aec5-48f9-9edd-bca5e7449020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:07.444227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e413ca-aec5-48f9-9edd-bca5e7449020 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:07.444516 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e413ca-aec5-48f9-9edd-bca5e7449020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:07.444792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e413ca-aec5-48f9-9edd-bca5e7449020 admin admin] PATH_INFO: `/v3/projects/3aea866bc8a5450da4debddfc37de132/users/3d8d9932b0a643038347439b91802aed/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:07.463104 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e413ca-aec5-48f9-9edd-bca5e7449020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aea866bc8a5450da4debddfc37de132, user_id=3d8d9932b0a643038347439b91802aed, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:07.465823 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e413ca-aec5-48f9-9edd-bca5e7449020 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:07.481477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1417/2820] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:13:07 2022] PUT /identity/v3/projects/3aea866bc8a5450da4debddfc37de132/users/3d8d9932b0a643038347439b91802aed/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:07.485635 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be66c5-04c4-4dac-a887-d5546321d39b admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:07.497065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be66c5-04c4-4dac-a887-d5546321d39b admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:07.558477 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39c9f9-f5e1-48d4-a486-37e137a9465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:07.559134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39c9f9-f5e1-48d4-a486-37e137a9465b admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:07.559426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39c9f9-f5e1-48d4-a486-37e137a9465b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:07.559698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39c9f9-f5e1-48d4-a486-37e137a9465b admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:07.560421 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39c9f9-f5e1-48d4-a486-37e137a9465b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:07.562102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39c9f9-f5e1-48d4-a486-37e137a9465b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:07.563530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:07.578541 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1404/2821] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:13:07.580334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e413ca-aec5-48f9-9edd-bca5e7449020 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:07.593824 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e413ca-aec5-48f9-9edd-bca5e7449020 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:07.599642 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5effc1f-b6d7-4b35-880a-6f6d54e411bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:07.600868 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5effc1f-b6d7-4b35-880a-6f6d54e411bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:07.601239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5effc1f-b6d7-4b35-880a-6f6d54e411bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:07.601641 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5effc1f-b6d7-4b35-880a-6f6d54e411bb admin admin] PATH_INFO: `/v3/projects/3aea866bc8a5450da4debddfc37de132/users/3d8d9932b0a643038347439b91802aed/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:07.603683 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5effc1f-b6d7-4b35-880a-6f6d54e411bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aea866bc8a5450da4debddfc37de132, user_id=3d8d9932b0a643038347439b91802aed, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:07.606944 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5effc1f-b6d7-4b35-880a-6f6d54e411bb admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:07.629893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1418/2822] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:13:07 2022] PUT /identity/v3/projects/3aea866bc8a5450da4debddfc37de132/users/3d8d9932b0a643038347439b91802aed/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:08.078521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a5a2a-187e-4816-8536-4f448a630749 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:08.078521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a5a2a-187e-4816-8536-4f448a630749 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:08.078757 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a5a2a-187e-4816-8536-4f448a630749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:08.106507 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-846a5a2a-187e-4816-8536-4f448a630749 None None] MFA Rules not processed for user `6816e6749679401cbfdd6cc8bcf050e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:08.108045 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-846a5a2a-187e-4816-8536-4f448a630749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:08.193622 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1405/2823] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:09.625261 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5effc1f-b6d7-4b35-880a-6f6d54e411bb admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:09.627650 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b5effc1f-b6d7-4b35-880a-6f6d54e411bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:09.724426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5effc1f-b6d7-4b35-880a-6f6d54e411bb admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:09.734313 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c289fcc-dbfb-495d-8d14-b9be6ae19f80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:09.739606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c289fcc-dbfb-495d-8d14-b9be6ae19f80 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:09.740874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c289fcc-dbfb-495d-8d14-b9be6ae19f80 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:09.741355 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c289fcc-dbfb-495d-8d14-b9be6ae19f80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:09.745263 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-4c289fcc-dbfb-495d-8d14-b9be6ae19f80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:09.759661 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c289fcc-dbfb-495d-8d14-b9be6ae19f80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:09.761288 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c289fcc-dbfb-495d-8d14-b9be6ae19f80 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:09.772386 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1419/2824] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:13:10.968989 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a5a2a-187e-4816-8536-4f448a630749 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:10.971130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-846a5a2a-187e-4816-8536-4f448a630749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:11.084059 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a5a2a-187e-4816-8536-4f448a630749 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:11.099801 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:11.100740 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:11.103514 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:11.103844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] PATH_INFO: `/v3/users/4ad8d2d3e04b4e06bff2c64bff5a1ac3` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:11.108185 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ad8d2d3e04b4e06bff2c64bff5a1ac3)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:11.113156 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:11.221426 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ad8d2d3e04b4e06bff2c64bff5a1ac3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:11.222534 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ad8d2d3e04b4e06bff2c64bff5a1ac3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:11.235603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ad8d2d3e04b4e06bff2c64bff5a1ac3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:11.237022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ad8d2d3e04b4e06bff2c64bff5a1ac3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:11.278858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-949743c4-46ea-4f95-b18c-ee30b0b49d4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ad8d2d3e04b4e06bff2c64bff5a1ac3'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:11.303887 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1406/2825] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:13:10 2022] DELETE /identity/v3/users/4ad8d2d3e04b4e06bff2c64bff5a1ac3 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:11.621367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c289fcc-dbfb-495d-8d14-b9be6ae19f80 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:11.654426 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c289fcc-dbfb-495d-8d14-b9be6ae19f80 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:11.773498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:11.775503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:11.775963 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:11.776458 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] PATH_INFO: `/v3/projects/24cf32bd40be4be9a83d0771e4f939fa` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:11.777848 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] RBAC: Authorizing `identity:delete_project(project_id=24cf32bd40be4be9a83d0771e4f939fa)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:11.780101 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:11.798758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:11.858146 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24cf32bd40be4be9a83d0771e4f939fa', 'request_id': 'req-0473679f-f483-4e73-a238-3114fda1887e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:11.868238 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24cf32bd40be4be9a83d0771e4f939fa', 'request_id': 'req-0473679f-f483-4e73-a238-3114fda1887e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:11.878388 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24cf32bd40be4be9a83d0771e4f939fa', 'request_id': 'req-0473679f-f483-4e73-a238-3114fda1887e'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:11.900936 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] The token cache is being invalidate because project 24cf32bd40be4be9a83d0771e4f939fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:13:11.901612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:11.903044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-0473679f-f483-4e73-a238-3114fda1887e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:11.906284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1420/2826] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:13:11 2022] DELETE /identity/v3/projects/24cf32bd40be4be9a83d0771e4f939fa => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:11.912008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d3226-428a-4e2a-bd0f-b333f0214921 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:11.912628 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d3226-428a-4e2a-bd0f-b333f0214921 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:11.913065 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d3226-428a-4e2a-bd0f-b333f0214921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:11.970679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b66d3226-428a-4e2a-bd0f-b333f0214921 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:11.973831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b66d3226-428a-4e2a-bd0f-b333f0214921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:12.000691 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:12.112742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1407/2827] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:12.121129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7afdc4-4045-4468-8634-01297ec0a0f8 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:12.121660 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7afdc4-4045-4468-8634-01297ec0a0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:12.122030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7afdc4-4045-4468-8634-01297ec0a0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:12.158707 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-db7afdc4-4045-4468-8634-01297ec0a0f8 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:12.160079 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-db7afdc4-4045-4468-8634-01297ec0a0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:12.180447 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1421/2828] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:12.186008 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d3226-428a-4e2a-bd0f-b333f0214921 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:12.203767 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d3226-428a-4e2a-bd0f-b333f0214921 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:12.223961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5df13b-642f-4c2b-9913-27a4544b05bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:12.225234 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df13b-642f-4c2b-9913-27a4544b05bb admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:12.228121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df13b-642f-4c2b-9913-27a4544b05bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:12.228121 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df13b-642f-4c2b-9913-27a4544b05bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:12.228952 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5df13b-642f-4c2b-9913-27a4544b05bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:13:12.229266 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5df13b-642f-4c2b-9913-27a4544b05bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:12.232222 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5df13b-642f-4c2b-9913-27a4544b05bb admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:12.252335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1408/2829] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:13:12.256620 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7afdc4-4045-4468-8634-01297ec0a0f8 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:12.270366 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7afdc4-4045-4468-8634-01297ec0a0f8 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:12.282685 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ffb2e-b31b-4b46-a353-c587accc301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:12.283592 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ffb2e-b31b-4b46-a353-c587accc301e admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:12.283731 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ffb2e-b31b-4b46-a353-c587accc301e admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:12.283972 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ffb2e-b31b-4b46-a353-c587accc301e admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:12.284686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ffb2e-b31b-4b46-a353-c587accc301e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:12.286601 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ffb2e-b31b-4b46-a353-c587accc301e admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:12.345412 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1422/2830] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:13:12 2022] POST /identity/v3/projects => generated 397 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:13:12.350071 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5df13b-642f-4c2b-9913-27a4544b05bb admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:12.374870 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5df13b-642f-4c2b-9913-27a4544b05bb admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:12.546099 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f53c7e-8c83-401c-a739-478206b4487c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:12.546969 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53c7e-8c83-401c-a739-478206b4487c admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:12.547418 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53c7e-8c83-401c-a739-478206b4487c admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:12.547809 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f53c7e-8c83-401c-a739-478206b4487c admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:12.548886 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f53c7e-8c83-401c-a739-478206b4487c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:12.551359 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f53c7e-8c83-401c-a739-478206b4487c admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:12.586596 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1409/2831] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:13:12 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:13:12.591469 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ffb2e-b31b-4b46-a353-c587accc301e admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:12.612878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ffb2e-b31b-4b46-a353-c587accc301e admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:12.628222 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec01a0-ec12-4b5a-82b2-c8885471e9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:12.629597 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec01a0-ec12-4b5a-82b2-c8885471e9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:12.630042 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec01a0-ec12-4b5a-82b2-c8885471e9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:12.630527 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec01a0-ec12-4b5a-82b2-c8885471e9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:12.631562 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec01a0-ec12-4b5a-82b2-c8885471e9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:12.633249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec01a0-ec12-4b5a-82b2-c8885471e9b4 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:12.657012 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1423/2832] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:13:12.662145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f53c7e-8c83-401c-a739-478206b4487c admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:12.687226 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f53c7e-8c83-401c-a739-478206b4487c admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:12.702973 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d594d2-0893-4b9d-8f65-b1dd26ed213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:12.703961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d594d2-0893-4b9d-8f65-b1dd26ed213e admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:12.706595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d594d2-0893-4b9d-8f65-b1dd26ed213e admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:12.707497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d594d2-0893-4b9d-8f65-b1dd26ed213e admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:12.708580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d594d2-0893-4b9d-8f65-b1dd26ed213e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:12.710530 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d594d2-0893-4b9d-8f65-b1dd26ed213e admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:12.726934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1410/2833] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:13:12.731770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec01a0-ec12-4b5a-82b2-c8885471e9b4 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:12.750696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec01a0-ec12-4b5a-82b2-c8885471e9b4 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:12.758563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734a3da-6396-40a2-a90c-a98b25de0ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:12.759930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734a3da-6396-40a2-a90c-a98b25de0ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:12.760229 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734a3da-6396-40a2-a90c-a98b25de0ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:12.760929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734a3da-6396-40a2-a90c-a98b25de0ca2 admin admin] PATH_INFO: `/v3/projects/fb4a420314f9405a960356f56fc646f1/users/a4ed56f1f5b9439eacf2ef715c97239b/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:12.789933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734a3da-6396-40a2-a90c-a98b25de0ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4a420314f9405a960356f56fc646f1, user_id=a4ed56f1f5b9439eacf2ef715c97239b, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:12.797821 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734a3da-6396-40a2-a90c-a98b25de0ca2 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:12.826513 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1424/2834] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:13:12 2022] PUT /identity/v3/projects/fb4a420314f9405a960356f56fc646f1/users/a4ed56f1f5b9439eacf2ef715c97239b/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:12.842508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27314ec4-e1f8-4190-bffa-13f7fa69dc1e None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:12.843819 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27314ec4-e1f8-4190-bffa-13f7fa69dc1e None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:12.844344 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27314ec4-e1f8-4190-bffa-13f7fa69dc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:12.880883 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-27314ec4-e1f8-4190-bffa-13f7fa69dc1e None None] MFA Rules not processed for user `a4ed56f1f5b9439eacf2ef715c97239b`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:12.882931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-27314ec4-e1f8-4190-bffa-13f7fa69dc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:12.977281 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1411/2835] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:13.002413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734a3da-6396-40a2-a90c-a98b25de0ca2 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:13.004703 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b734a3da-6396-40a2-a90c-a98b25de0ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:13.095056 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734a3da-6396-40a2-a90c-a98b25de0ca2 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:13.100310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97211e-1408-43e2-9aae-a860c907544c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:13.101822 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97211e-1408-43e2-9aae-a860c907544c service cinder] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:13.102477 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97211e-1408-43e2-9aae-a860c907544c service cinder] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:13.103283 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97211e-1408-43e2-9aae-a860c907544c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:13.117010 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97211e-1408-43e2-9aae-a860c907544c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:13.120129 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97211e-1408-43e2-9aae-a860c907544c service cinder] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:13.133402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1425/2836] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:13:14.234462 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f009b4-3d19-4c08-8807-d96ee9844cb5 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:14.234942 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f009b4-3d19-4c08-8807-d96ee9844cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:14.235026 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f009b4-3d19-4c08-8807-d96ee9844cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:14.280743 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-80f009b4-3d19-4c08-8807-d96ee9844cb5 None None] MFA Rules not processed for user `3d8d9932b0a643038347439b91802aed`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:14.283392 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-80f009b4-3d19-4c08-8807-d96ee9844cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:14.289707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:14.412303 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1412/2837] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:14.421093 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97211e-1408-43e2-9aae-a860c907544c service cinder] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:14.422606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-1b97211e-1408-43e2-9aae-a860c907544c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:14.479743 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:14.550041 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97211e-1408-43e2-9aae-a860c907544c service cinder] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:14.556178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad66e2-8032-4688-b08a-e81c0704a4d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:14.557008 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad66e2-8032-4688-b08a-e81c0704a4d5 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:14.557295 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad66e2-8032-4688-b08a-e81c0704a4d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:14.557608 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad66e2-8032-4688-b08a-e81c0704a4d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:14.559503 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9bad66e2-8032-4688-b08a-e81c0704a4d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:14.648565 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad66e2-8032-4688-b08a-e81c0704a4d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:14.650378 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad66e2-8032-4688-b08a-e81c0704a4d5 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:14.677812 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1426/2838] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:13:15.986631 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de010d5-4e23-41cc-834a-7bfa5efa7691 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:15.987085 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de010d5-4e23-41cc-834a-7bfa5efa7691 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:15.987287 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de010d5-4e23-41cc-834a-7bfa5efa7691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:16.006200 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4de010d5-4e23-41cc-834a-7bfa5efa7691 None None] MFA Rules not processed for user `5aa8001294eb45179e9b51a8974beb8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:16.007471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4de010d5-4e23-41cc-834a-7bfa5efa7691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:16.015145 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1413/2839] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:20.581795 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad66e2-8032-4688-b08a-e81c0704a4d5 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:20.647532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad66e2-8032-4688-b08a-e81c0704a4d5 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:20.652345 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:20.653219 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:20.653505 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:20.653784 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] PATH_INFO: `/v3/users/a4ed56f1f5b9439eacf2ef715c97239b` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:20.655249 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4ed56f1f5b9439eacf2ef715c97239b)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:20.657255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:20.704740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4ed56f1f5b9439eacf2ef715c97239b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:20.718400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4ed56f1f5b9439eacf2ef715c97239b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:20.726401 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4ed56f1f5b9439eacf2ef715c97239b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:20.727161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4ed56f1f5b9439eacf2ef715c97239b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:20.732858 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:20.735024 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4ed56f1f5b9439eacf2ef715c97239b'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:20.742540 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1427/2840] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:13:20 2022] DELETE /identity/v3/users/a4ed56f1f5b9439eacf2ef715c97239b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:20.747076 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ebc02-7a7f-4be7-91f6-efe7e05f64d9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:20.747565 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ebc02-7a7f-4be7-91f6-efe7e05f64d9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:20.747669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ebc02-7a7f-4be7-91f6-efe7e05f64d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:20.768464 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-d56ebc02-7a7f-4be7-91f6-efe7e05f64d9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:20.769588 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d56ebc02-7a7f-4be7-91f6-efe7e05f64d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:20.829889 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1414/2841] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:20.840049 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:20.841453 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:20.899798 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa5eea-bd9a-45fc-932d-0fd6c2417f42 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:20.903489 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f7762-b05c-41ba-b015-0a9069bcd77b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:20.904150 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f7762-b05c-41ba-b015-0a9069bcd77b service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:20.904465 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f7762-b05c-41ba-b015-0a9069bcd77b service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:20.904730 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f7762-b05c-41ba-b015-0a9069bcd77b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:20.914344 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f7762-b05c-41ba-b015-0a9069bcd77b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:20.915810 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f7762-b05c-41ba-b015-0a9069bcd77b service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:20.925227 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1428/2842] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:13:21.259140 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ebc02-7a7f-4be7-91f6-efe7e05f64d9 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:21.260879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-d56ebc02-7a7f-4be7-91f6-efe7e05f64d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:21.275032 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ebc02-7a7f-4be7-91f6-efe7e05f64d9 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:21.280721 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:21.281765 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:21.282054 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:21.282338 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] PATH_INFO: `/v3/projects/fb4a420314f9405a960356f56fc646f1` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:21.283603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4a420314f9405a960356f56fc646f1)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:21.286395 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:21.333444 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4a420314f9405a960356f56fc646f1', 'request_id': 'req-92f18631-a667-4568-b5a9-9a90de762de6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:21.338763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4a420314f9405a960356f56fc646f1', 'request_id': 'req-92f18631-a667-4568-b5a9-9a90de762de6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:21.345860 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4a420314f9405a960356f56fc646f1', 'request_id': 'req-92f18631-a667-4568-b5a9-9a90de762de6'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:21.358472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] The token cache is being invalidate because project fb4a420314f9405a960356f56fc646f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:13:21.358773 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:21.359391 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:21.361173 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1415/2843] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:13:21 2022] DELETE /identity/v3/projects/fb4a420314f9405a960356f56fc646f1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:29.573304 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ff7e3-f7a9-4b08-a79c-d400691b670c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:29.573794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ff7e3-f7a9-4b08-a79c-d400691b670c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:29.573893 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ff7e3-f7a9-4b08-a79c-d400691b670c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:29.591933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-790ff7e3-f7a9-4b08-a79c-d400691b670c None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:29.593122 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-790ff7e3-f7a9-4b08-a79c-d400691b670c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:29.650637 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1429/2844] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:29.697475 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:29.698882 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:29.719239 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:29.771844 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f18631-a667-4568-b5a9-9a90de762de6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:29.776812 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f15f3-0eb0-4a6b-9beb-a665dfe5c0cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:29.777461 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f15f3-0eb0-4a6b-9beb-a665dfe5c0cb service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:29.777744 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f15f3-0eb0-4a6b-9beb-a665dfe5c0cb service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:29.778019 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f15f3-0eb0-4a6b-9beb-a665dfe5c0cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:29.786439 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f15f3-0eb0-4a6b-9beb-a665dfe5c0cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:29.788061 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f15f3-0eb0-4a6b-9beb-a665dfe5c0cb service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:29.796022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1416/2845] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:13:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:13:30.306119 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376d92-49db-47d8-9c06-7c252cfb9c05 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:30.306418 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376d92-49db-47d8-9c06-7c252cfb9c05 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:30.306733 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1376d92-49db-47d8-9c06-7c252cfb9c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:30.324863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-d1376d92-49db-47d8-9c06-7c252cfb9c05 None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:30.326067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-d1376d92-49db-47d8-9c06-7c252cfb9c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:30.333920 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1430/2846] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 26 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:13:30.345677 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f15f3-0eb0-4a6b-9beb-a665dfe5c0cb service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:30.356113 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f15f3-0eb0-4a6b-9beb-a665dfe5c0cb service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:30.361029 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c7140-3a18-4b79-8da2-40f815ca5a87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:30.361650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c7140-3a18-4b79-8da2-40f815ca5a87 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:30.361931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c7140-3a18-4b79-8da2-40f815ca5a87 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:30.362206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c7140-3a18-4b79-8da2-40f815ca5a87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:30.370363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c7140-3a18-4b79-8da2-40f815ca5a87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:30.371970 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c7140-3a18-4b79-8da2-40f815ca5a87 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:30.380380 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1417/2847] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:13:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:13:30.564611 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb732-7b63-49be-b9ae-be8f7c0a42ab None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:30.564916 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb732-7b63-49be-b9ae-be8f7c0a42ab None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:30.565177 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cb732-7b63-49be-b9ae-be8f7c0a42ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:30.583179 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b12cb732-7b63-49be-b9ae-be8f7c0a42ab None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:30.584359 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b12cb732-7b63-49be-b9ae-be8f7c0a42ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:30.591385 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1431/2848] 10.222.0.57 () {58 vars in 1009 bytes} [Thu May 26 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:13:30.603116 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c7140-3a18-4b79-8da2-40f815ca5a87 service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:30.613539 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c7140-3a18-4b79-8da2-40f815ca5a87 service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:30.618715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a08ec-7bb6-4525-8e3e-ad697beace5c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:30.619341 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a08ec-7bb6-4525-8e3e-ad697beace5c service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:30.619624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a08ec-7bb6-4525-8e3e-ad697beace5c service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:30.619900 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a08ec-7bb6-4525-8e3e-ad697beace5c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:30.628279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a08ec-7bb6-4525-8e3e-ad697beace5c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:30.629860 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a08ec-7bb6-4525-8e3e-ad697beace5c service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:30.637775 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1418/2849] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:13:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:13:30.948564 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4cb85-e0fe-4abf-a784-436578f3e41e None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:30.948852 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4cb85-e0fe-4abf-a784-436578f3e41e None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:30.949130 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f4cb85-e0fe-4abf-a784-436578f3e41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:30.966239 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-e8f4cb85-e0fe-4abf-a784-436578f3e41e None None] MFA Rules not processed for user `e67f497e8a8b46e1b665999ba8fda38d`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:30.967463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e8f4cb85-e0fe-4abf-a784-436578f3e41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:30.974633 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1432/2850] 10.222.0.57 () {58 vars in 967 bytes} [Thu May 26 21:13:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:30.986841 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a08ec-7bb6-4525-8e3e-ad697beace5c service swift] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:30.997563 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a08ec-7bb6-4525-8e3e-ad697beace5c service swift] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:31.003747 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f890354-d1b2-49c7-b6e2-180937599a31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1285406084604bd19287de9e1beeeaa8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:31.004749 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f890354-d1b2-49c7-b6e2-180937599a31 service swift] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:31.005201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f890354-d1b2-49c7-b6e2-180937599a31 service swift] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:31.005657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f890354-d1b2-49c7-b6e2-180937599a31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:31.014879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f890354-d1b2-49c7-b6e2-180937599a31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:31.016472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f890354-d1b2-49c7-b6e2-180937599a31 service swift] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:31.024472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1419/2851] 10.222.0.57 () {60 vars in 1384 bytes} [Thu May 26 21:13:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:13:33.071463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4cb85-e0fe-4abf-a784-436578f3e41e None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:33.073030 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-e8f4cb85-e0fe-4abf-a784-436578f3e41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:33.176647 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f4cb85-e0fe-4abf-a784-436578f3e41e None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:33.181749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:33.183596 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:33.184005 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:33.184360 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] PATH_INFO: `/v3/users/6816e6749679401cbfdd6cc8bcf050e5` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:33.185663 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6816e6749679401cbfdd6cc8bcf050e5)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:33.189217 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:33.225770 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:33.250496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6816e6749679401cbfdd6cc8bcf050e5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:33.262968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6816e6749679401cbfdd6cc8bcf050e5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:33.270752 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6816e6749679401cbfdd6cc8bcf050e5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:33.271380 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6816e6749679401cbfdd6cc8bcf050e5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:33.278407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6816e6749679401cbfdd6cc8bcf050e5'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:33.285495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1433/2852] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:13:33 2022] DELETE /identity/v3/users/6816e6749679401cbfdd6cc8bcf050e5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:33.291142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce237414-1cbe-4074-8eef-901efc2b0d05 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:33.291614 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce237414-1cbe-4074-8eef-901efc2b0d05 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:33.292021 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce237414-1cbe-4074-8eef-901efc2b0d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:33.311845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-ce237414-1cbe-4074-8eef-901efc2b0d05 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:13:33.313734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ce237414-1cbe-4074-8eef-901efc2b0d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:33.377102 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1420/2853] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:13:33.387192 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:33.388681 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:33.449106 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95000c91-e0ba-4981-9854-9f376c24f4a8 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:33.454063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c01e8-bfc1-46fb-96dd-e38851d12a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:33.454759 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c01e8-bfc1-46fb-96dd-e38851d12a63 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:33.455047 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c01e8-bfc1-46fb-96dd-e38851d12a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:33.455391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c01e8-bfc1-46fb-96dd-e38851d12a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:33.467168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c01e8-bfc1-46fb-96dd-e38851d12a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:33.469159 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c01e8-bfc1-46fb-96dd-e38851d12a63 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:33.479762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1434/2854] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:13:33.658728 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce237414-1cbe-4074-8eef-901efc2b0d05 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:33.660263 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ce237414-1cbe-4074-8eef-901efc2b0d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:33.675138 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce237414-1cbe-4074-8eef-901efc2b0d05 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:33.682419 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:33.683871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:33.684415 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:33.684965 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] PATH_INFO: `/v3/projects/2ac6e23534754c618cebf75ae3e36eb6` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:33.686846 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ac6e23534754c618cebf75ae3e36eb6)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:33.689373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:33.742763 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ac6e23534754c618cebf75ae3e36eb6', 'request_id': 'req-431cb167-af36-4315-b7ca-e986905b4c26'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:33.748267 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ac6e23534754c618cebf75ae3e36eb6', 'request_id': 'req-431cb167-af36-4315-b7ca-e986905b4c26'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:33.755499 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ac6e23534754c618cebf75ae3e36eb6', 'request_id': 'req-431cb167-af36-4315-b7ca-e986905b4c26'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:33.757619 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:33.771476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] The token cache is being invalidate because project 2ac6e23534754c618cebf75ae3e36eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80269) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:13:33.771764 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:33.772409 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:13:33.773950 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1421/2855] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:13:33 2022] DELETE /identity/v3/projects/2ac6e23534754c618cebf75ae3e36eb6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:13:43.474741 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:DELETE=6,keystone:SELECT=45,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:43.768646 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:13:48.528836 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c01e8-bfc1-46fb-96dd-e38851d12a63 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:48.530294 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-035c01e8-bfc1-46fb-96dd-e38851d12a63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:48.541702 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c01e8-bfc1-46fb-96dd-e38851d12a63 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:48.589908 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc560d2-0029-4915-80c3-4a72123f3d90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:48.590512 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc560d2-0029-4915-80c3-4a72123f3d90 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:48.590806 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc560d2-0029-4915-80c3-4a72123f3d90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:48.591086 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc560d2-0029-4915-80c3-4a72123f3d90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:48.592895 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fcc560d2-0029-4915-80c3-4a72123f3d90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:48.660434 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc560d2-0029-4915-80c3-4a72123f3d90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:48.661876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc560d2-0029-4915-80c3-4a72123f3d90 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:48.671381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1435/2856] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:48 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:13:50.387632 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:13:50.389195 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:50.464496 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cb167-af36-4315-b7ca-e986905b4c26 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:13:50.470043 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05671565-8f1d-4c03-a886-48d51b3b36fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:13:50.470746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05671565-8f1d-4c03-a886-48d51b3b36fe service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:13:50.471031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05671565-8f1d-4c03-a886-48d51b3b36fe service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:13:50.471308 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05671565-8f1d-4c03-a886-48d51b3b36fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:13:50.473112 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-05671565-8f1d-4c03-a886-48d51b3b36fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:13:50.486864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05671565-8f1d-4c03-a886-48d51b3b36fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:13:50.488522 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05671565-8f1d-4c03-a886-48d51b3b36fe service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:13:50.501262 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1422/2857] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:13:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:13:58.667116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=50 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:14:00.495580 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=34 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:15:38.065357 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc560d2-0029-4915-80c3-4a72123f3d90 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:15:38.066790 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-fcc560d2-0029-4915-80c3-4a72123f3d90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:15:38.077497 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc560d2-0029-4915-80c3-4a72123f3d90 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:15:38.081087 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be17431-7e44-4ce3-b20c-1693b9cf2d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:15:38.081727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be17431-7e44-4ce3-b20c-1693b9cf2d79 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:15:38.082015 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be17431-7e44-4ce3-b20c-1693b9cf2d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:15:38.082310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be17431-7e44-4ce3-b20c-1693b9cf2d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:15:38.084040 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-5be17431-7e44-4ce3-b20c-1693b9cf2d79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:15:38.094022 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be17431-7e44-4ce3-b20c-1693b9cf2d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:15:38.095571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be17431-7e44-4ce3-b20c-1693b9cf2d79 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:15:38.104084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1436/2858] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:15:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:15:44.753899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05671565-8f1d-4c03-a886-48d51b3b36fe service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:15:44.755385 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-05671565-8f1d-4c03-a886-48d51b3b36fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:15:44.766769 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05671565-8f1d-4c03-a886-48d51b3b36fe service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:15:44.770550 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76771d6e-60ea-4cc5-9d02-f4dd5d2be6a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:15:44.771186 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76771d6e-60ea-4cc5-9d02-f4dd5d2be6a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:15:44.771471 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76771d6e-60ea-4cc5-9d02-f4dd5d2be6a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:15:44.771750 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76771d6e-60ea-4cc5-9d02-f4dd5d2be6a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:15:44.773932 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-76771d6e-60ea-4cc5-9d02-f4dd5d2be6a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:15:44.791010 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76771d6e-60ea-4cc5-9d02-f4dd5d2be6a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:15:44.792585 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76771d6e-60ea-4cc5-9d02-f4dd5d2be6a0 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:15:44.805339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1423/2859] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:15:45.974340 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be17431-7e44-4ce3-b20c-1693b9cf2d79 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:15:45.984262 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be17431-7e44-4ce3-b20c-1693b9cf2d79 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:15:45.987878 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c8a65-6d10-4359-a108-22344c712b8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:15:45.988696 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c8a65-6d10-4359-a108-22344c712b8a service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:15:45.988839 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c8a65-6d10-4359-a108-22344c712b8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:15:45.989410 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c8a65-6d10-4359-a108-22344c712b8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:15:45.991406 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f50c8a65-6d10-4359-a108-22344c712b8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:15:46.007168 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c8a65-6d10-4359-a108-22344c712b8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:15:46.008758 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c8a65-6d10-4359-a108-22344c712b8a service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:15:46.019808 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1437/2860] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:15:54.798816 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:15:56.014792 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=12 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:15:59.007294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76771d6e-60ea-4cc5-9d02-f4dd5d2be6a0 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:15:59.016033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76771d6e-60ea-4cc5-9d02-f4dd5d2be6a0 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:15:59.019707 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ba17a-7a6d-4d05-9738-5ecf5e6d992c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:15:59.020336 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ba17a-7a6d-4d05-9738-5ecf5e6d992c service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:15:59.020621 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ba17a-7a6d-4d05-9738-5ecf5e6d992c service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:15:59.020899 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ba17a-7a6d-4d05-9738-5ecf5e6d992c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:15:59.023151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-648ba17a-7a6d-4d05-9738-5ecf5e6d992c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:15:59.032601 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ba17a-7a6d-4d05-9738-5ecf5e6d992c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:15:59.034033 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ba17a-7a6d-4d05-9738-5ecf5e6d992c service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:15:59.042681 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1424/2861] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:15:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:15:59.142566 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c8a65-6d10-4359-a108-22344c712b8a service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:15:59.151723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c8a65-6d10-4359-a108-22344c712b8a service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:15:59.155407 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557bc18-386f-44d5-a065-24656f2d4e11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:15:59.156043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557bc18-386f-44d5-a065-24656f2d4e11 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:15:59.156330 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557bc18-386f-44d5-a065-24656f2d4e11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:15:59.156612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557bc18-386f-44d5-a065-24656f2d4e11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:15:59.158340 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b557bc18-386f-44d5-a065-24656f2d4e11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:15:59.169068 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557bc18-386f-44d5-a065-24656f2d4e11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:15:59.170718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557bc18-386f-44d5-a065-24656f2d4e11 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:15:59.179298 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1438/2862] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:15:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:15:59.274294 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ba17a-7a6d-4d05-9738-5ecf5e6d992c service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:15:59.282995 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ba17a-7a6d-4d05-9738-5ecf5e6d992c service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:15:59.286603 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8528148-55b3-4de9-881f-2a3950143b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:15:59.287243 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528148-55b3-4de9-881f-2a3950143b9c service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:15:59.287525 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528148-55b3-4de9-881f-2a3950143b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:15:59.287798 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528148-55b3-4de9-881f-2a3950143b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:15:59.290095 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-e8528148-55b3-4de9-881f-2a3950143b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:15:59.304096 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8528148-55b3-4de9-881f-2a3950143b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:15:59.305529 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8528148-55b3-4de9-881f-2a3950143b9c service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:15:59.316295 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1425/2863] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:15:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:15:59.375995 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557bc18-386f-44d5-a065-24656f2d4e11 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:15:59.377382 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b557bc18-386f-44d5-a065-24656f2d4e11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:15:59.446478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557bc18-386f-44d5-a065-24656f2d4e11 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:15:59.451929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fbc19-cd51-4032-a033-3fc58370e1dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:15:59.452586 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fbc19-cd51-4032-a033-3fc58370e1dd service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:15:59.452873 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fbc19-cd51-4032-a033-3fc58370e1dd service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:15:59.453155 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fbc19-cd51-4032-a033-3fc58370e1dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:15:59.454961 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-336fbc19-cd51-4032-a033-3fc58370e1dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:15:59.467055 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fbc19-cd51-4032-a033-3fc58370e1dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:15:59.468647 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fbc19-cd51-4032-a033-3fc58370e1dd service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:15:59.478720 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1439/2864] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:15:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:16:00.920657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8528148-55b3-4de9-881f-2a3950143b9c service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:16:00.934147 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8528148-55b3-4de9-881f-2a3950143b9c service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:16:00.940149 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf7f72-7372-49d5-8419-3f53cb5aa99f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:16:00.940825 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf7f72-7372-49d5-8419-3f53cb5aa99f service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:16:00.941117 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf7f72-7372-49d5-8419-3f53cb5aa99f service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:16:00.941388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf7f72-7372-49d5-8419-3f53cb5aa99f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:16:00.943215 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5caf7f72-7372-49d5-8419-3f53cb5aa99f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:16:00.956135 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf7f72-7372-49d5-8419-3f53cb5aa99f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:16:00.957731 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf7f72-7372-49d5-8419-3f53cb5aa99f service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:16:00.968833 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1426/2865] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:16:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:16:09.474285 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=38 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:16:10.963999 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=18 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:25.129874 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fbc19-cd51-4032-a033-3fc58370e1dd service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:25.201174 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fbc19-cd51-4032-a033-3fc58370e1dd service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:25.206970 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87e28e-529b-4a6e-8803-65afe670f7d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:25.207624 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87e28e-529b-4a6e-8803-65afe670f7d9 service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:25.207909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87e28e-529b-4a6e-8803-65afe670f7d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:25.208184 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87e28e-529b-4a6e-8803-65afe670f7d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:25.210639 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e87e28e-529b-4a6e-8803-65afe670f7d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:25.243931 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87e28e-529b-4a6e-8803-65afe670f7d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:25.245498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87e28e-529b-4a6e-8803-65afe670f7d9 service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:25.256535 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1440/2866] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:17:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:17:29.626483 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf7f72-7372-49d5-8419-3f53cb5aa99f service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:29.692542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf7f72-7372-49d5-8419-3f53cb5aa99f service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:29.699297 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8662a-5588-453c-9136-821cd1f27496 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:29.699956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8662a-5588-453c-9136-821cd1f27496 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:29.700240 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8662a-5588-453c-9136-821cd1f27496 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:29.700518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8662a-5588-453c-9136-821cd1f27496 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:29.702279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b4e8662a-5588-453c-9136-821cd1f27496 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:29.727412 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8662a-5588-453c-9136-821cd1f27496 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:29.729013 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8662a-5588-453c-9136-821cd1f27496 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:29.739369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1427/2867] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:17:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:17:32.615134 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87e28e-529b-4a6e-8803-65afe670f7d9 service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:32.616719 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-9e87e28e-529b-4a6e-8803-65afe670f7d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:32.636235 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87e28e-529b-4a6e-8803-65afe670f7d9 service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:32.641909 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1b2c5-40b5-40f0-8b7b-13b8a5c9a7d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:32.642998 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1b2c5-40b5-40f0-8b7b-13b8a5c9a7d4 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:32.643534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1b2c5-40b5-40f0-8b7b-13b8a5c9a7d4 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:32.644189 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1b2c5-40b5-40f0-8b7b-13b8a5c9a7d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:32.647301 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-abf1b2c5-40b5-40f0-8b7b-13b8a5c9a7d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:32.663071 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1b2c5-40b5-40f0-8b7b-13b8a5c9a7d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:32.665136 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1b2c5-40b5-40f0-8b7b-13b8a5c9a7d4 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:32.703943 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1441/2868] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:17:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:17:32.712388 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8662a-5588-453c-9136-821cd1f27496 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:32.714251 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b4e8662a-5588-453c-9136-821cd1f27496 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:32.834206 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8662a-5588-453c-9136-821cd1f27496 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:32.838771 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:32.839638 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:32.839922 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:32.840201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] PATH_INFO: `/v3/users/5aa8001294eb45179e9b51a8974beb8a` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:32.841382 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aa8001294eb45179e9b51a8974beb8a)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:32.843408 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:32.889911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aa8001294eb45179e9b51a8974beb8a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:32.890497 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aa8001294eb45179e9b51a8974beb8a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:32.897404 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aa8001294eb45179e9b51a8974beb8a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:32.898046 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aa8001294eb45179e9b51a8974beb8a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:32.905082 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:32.911056 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aa8001294eb45179e9b51a8974beb8a'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:32.924404 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1428/2869] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:17:32 2022] DELETE /identity/v3/users/5aa8001294eb45179e9b51a8974beb8a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:17:33.095983 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1b2c5-40b5-40f0-8b7b-13b8a5c9a7d4 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:33.108131 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1b2c5-40b5-40f0-8b7b-13b8a5c9a7d4 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:33.182250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:33.183043 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:33.183325 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:33.183602 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] PATH_INFO: `/v3/projects/759c767b215942e5852e12fa01e22001` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:33.184774 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] RBAC: Authorizing `identity:delete_project(project_id=759c767b215942e5852e12fa01e22001)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:33.186709 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:33.231004 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '759c767b215942e5852e12fa01e22001', 'request_id': 'req-54fd265e-66ef-4bed-b04e-362a69f1b76d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.237044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '759c767b215942e5852e12fa01e22001', 'request_id': 'req-54fd265e-66ef-4bed-b04e-362a69f1b76d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.243400 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '759c767b215942e5852e12fa01e22001', 'request_id': 'req-54fd265e-66ef-4bed-b04e-362a69f1b76d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.245020 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:33.256200 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] The token cache is being invalidate because project 759c767b215942e5852e12fa01e22001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:17:33.256485 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.257096 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.258674 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1442/2870] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:17:33 2022] DELETE /identity/v3/projects/759c767b215942e5852e12fa01e22001 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:17:33.262634 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:33.264022 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:33.276593 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89911f2d-9ddf-417f-a880-944c718baef6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:33.332323 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:33.333182 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:33.333466 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:33.333742 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] PATH_INFO: `/v3/users/3d8d9932b0a643038347439b91802aed` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:33.334941 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d8d9932b0a643038347439b91802aed)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:33.336864 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:33.381476 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d8d9932b0a643038347439b91802aed'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.382111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d8d9932b0a643038347439b91802aed'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.388362 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d8d9932b0a643038347439b91802aed'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.389025 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d8d9932b0a643038347439b91802aed'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.400655 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-caee79aa-a5b1-4b03-a414-7613329da5b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d8d9932b0a643038347439b91802aed'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.413094 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1429/2871] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:17:33 2022] DELETE /identity/v3/users/3d8d9932b0a643038347439b91802aed => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:17:33.591865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:33.593649 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:33.607884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd265e-66ef-4bed-b04e-362a69f1b76d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:33.670827 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:33.671698 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:33.671989 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:33.672271 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] PATH_INFO: `/v3/projects/3aea866bc8a5450da4debddfc37de132` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:33.673462 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aea866bc8a5450da4debddfc37de132)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:33.675480 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:33.721496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aea866bc8a5450da4debddfc37de132', 'request_id': 'req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.728100 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aea866bc8a5450da4debddfc37de132', 'request_id': 'req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.733534 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aea866bc8a5450da4debddfc37de132', 'request_id': 'req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.745727 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] The token cache is being invalidate because project 3aea866bc8a5450da4debddfc37de132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:17:33.746014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.746665 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-ee146fd3-4d84-4e27-a3ae-1f6c1e3af030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:17:33.748194 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1443/2872] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:17:33 2022] DELETE /identity/v3/projects/3aea866bc8a5450da4debddfc37de132 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:17:33.753137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a4b54-804e-4b71-ba89-70ec228b8978 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:33.753590 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a4b54-804e-4b71-ba89-70ec228b8978 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:33.753700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a4b54-804e-4b71-ba89-70ec228b8978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:33.810424 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-a41a4b54-804e-4b71-ba89-70ec228b8978 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:17:33.811679 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-a41a4b54-804e-4b71-ba89-70ec228b8978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:33.839795 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:33.877734 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1430/2873] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:17:33.885875 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850a81d-b6ed-4134-ac19-d008dd18477a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:33.886160 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850a81d-b6ed-4134-ac19-d008dd18477a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:33.886438 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b850a81d-b6ed-4134-ac19-d008dd18477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:33.905258 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-b850a81d-b6ed-4134-ac19-d008dd18477a None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:17:33.906463 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-b850a81d-b6ed-4134-ac19-d008dd18477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:33.914571 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1444/2874] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:17:33.918669 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a4b54-804e-4b71-ba89-70ec228b8978 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:33.928167 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a4b54-804e-4b71-ba89-70ec228b8978 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:33.932678 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540080b-7bce-4735-8aaf-b7da6ab5a9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:33.933317 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540080b-7bce-4735-8aaf-b7da6ab5a9ab admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:33.933597 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540080b-7bce-4735-8aaf-b7da6ab5a9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:33.933871 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540080b-7bce-4735-8aaf-b7da6ab5a9ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:33.934626 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540080b-7bce-4735-8aaf-b7da6ab5a9ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80269) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:17:33.934931 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540080b-7bce-4735-8aaf-b7da6ab5a9ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:33.936526 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540080b-7bce-4735-8aaf-b7da6ab5a9ab admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:33.948137 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1431/2875] 10.222.0.57 () {58 vars in 1152 bytes} [Thu May 26 21:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:17:33.952092 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850a81d-b6ed-4134-ac19-d008dd18477a None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:33.961748 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b850a81d-b6ed-4134-ac19-d008dd18477a None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:33.966322 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546acc-4901-4bf1-a25a-276bda097607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:33.966991 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546acc-4901-4bf1-a25a-276bda097607 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:33.967277 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546acc-4901-4bf1-a25a-276bda097607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:33.967555 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546acc-4901-4bf1-a25a-276bda097607 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:33.968245 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2546acc-4901-4bf1-a25a-276bda097607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:33.970118 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2546acc-4901-4bf1-a25a-276bda097607 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:33.995113 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1445/2876] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:17:33 2022] POST /identity/v3/projects => generated 379 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:17:33.999051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540080b-7bce-4735-8aaf-b7da6ab5a9ab admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:34.009258 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540080b-7bce-4735-8aaf-b7da6ab5a9ab admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:34.062940 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d64e8-0a21-42af-9084-4335a872b6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:34.063591 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d64e8-0a21-42af-9084-4335a872b6d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:34.063876 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d64e8-0a21-42af-9084-4335a872b6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:34.064151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d64e8-0a21-42af-9084-4335a872b6d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:34.064834 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d64e8-0a21-42af-9084-4335a872b6d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:34.066780 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d64e8-0a21-42af-9084-4335a872b6d3 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:34.088250 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1432/2877] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:17:33 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:17:34.092374 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546acc-4901-4bf1-a25a-276bda097607 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:34.102782 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546acc-4901-4bf1-a25a-276bda097607 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:34.107381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5113c5-9e74-4d54-b8a2-7eddd2889982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:34.108036 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5113c5-9e74-4d54-b8a2-7eddd2889982 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:34.108324 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5113c5-9e74-4d54-b8a2-7eddd2889982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:34.108606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5113c5-9e74-4d54-b8a2-7eddd2889982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:34.109328 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5113c5-9e74-4d54-b8a2-7eddd2889982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:34.110985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5113c5-9e74-4d54-b8a2-7eddd2889982 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:34.121448 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1446/2878] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:17:34.125213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d64e8-0a21-42af-9084-4335a872b6d3 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:34.137516 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d64e8-0a21-42af-9084-4335a872b6d3 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:34.142157 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93850144-9777-4bc4-a246-d8e4601e1736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:34.142910 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93850144-9777-4bc4-a246-d8e4601e1736 admin admin] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:34.143201 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93850144-9777-4bc4-a246-d8e4601e1736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:34.143485 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93850144-9777-4bc4-a246-d8e4601e1736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:34.144217 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93850144-9777-4bc4-a246-d8e4601e1736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:34.145858 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93850144-9777-4bc4-a246-d8e4601e1736 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:34.156966 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1433/2879] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:17:34.161609 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5113c5-9e74-4d54-b8a2-7eddd2889982 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:34.173076 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5113c5-9e74-4d54-b8a2-7eddd2889982 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:34.177789 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3eb23-69d4-46e8-b46c-bf969c192561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:34.178713 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3eb23-69d4-46e8-b46c-bf969c192561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:34.179334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3eb23-69d4-46e8-b46c-bf969c192561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:34.179334 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3eb23-69d4-46e8-b46c-bf969c192561 admin admin] PATH_INFO: `/v3/projects/1b6678b588cc47f48b84d3120deccdb4/users/847b10367248482a8763e6b5d42f332c/roles/d3bdf57150744faf9c5c25c317df48fc` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:34.194417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3eb23-69d4-46e8-b46c-bf969c192561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6678b588cc47f48b84d3120deccdb4, user_id=847b10367248482a8763e6b5d42f332c, role_id=d3bdf57150744faf9c5c25c317df48fc)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:34.197102 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3eb23-69d4-46e8-b46c-bf969c192561 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:34.212367 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1447/2880] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:17:34 2022] PUT /identity/v3/projects/1b6678b588cc47f48b84d3120deccdb4/users/847b10367248482a8763e6b5d42f332c/roles/d3bdf57150744faf9c5c25c317df48fc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:17:34.217552 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3696f4-9bd4-4829-9d00-e8f859c6d4e9 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:34.217865 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3696f4-9bd4-4829-9d00-e8f859c6d4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:34.218130 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3696f4-9bd4-4829-9d00-e8f859c6d4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:34.235515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-4c3696f4-9bd4-4829-9d00-e8f859c6d4e9 None None] MFA Rules not processed for user `c2ba123046b04572883d482380082522`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:17:34.236706 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4c3696f4-9bd4-4829-9d00-e8f859c6d4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:34.295957 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1434/2881] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:17:34.305979 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3eb23-69d4-46e8-b46c-bf969c192561 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:34.307671 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-aff3eb23-69d4-46e8-b46c-bf969c192561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:34.329686 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:34.386778 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3eb23-69d4-46e8-b46c-bf969c192561 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:34.390417 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448fa57-be3c-4292-b530-717df1ae2473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:34.391082 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448fa57-be3c-4292-b530-717df1ae2473 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:34.391369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448fa57-be3c-4292-b530-717df1ae2473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:34.391653 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448fa57-be3c-4292-b530-717df1ae2473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:34.400498 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448fa57-be3c-4292-b530-717df1ae2473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:34.401923 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448fa57-be3c-4292-b530-717df1ae2473 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:34.411161 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1448/2882] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:17:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:17:38.192832 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b4a67-f9bc-43d4-8422-4a6084f6d8b0 None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:38.193330 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b4a67-f9bc-43d4-8422-4a6084f6d8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:38.193828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b4a67-f9bc-43d4-8422-4a6084f6d8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:38.229174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-b53b4a67-f9bc-43d4-8422-4a6084f6d8b0 None None] MFA Rules not processed for user `847b10367248482a8763e6b5d42f332c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:17:38.230307 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-b53b4a67-f9bc-43d4-8422-4a6084f6d8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:38.259521 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:38.305741 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1435/2883] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:17:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:17:38.310116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448fa57-be3c-4292-b530-717df1ae2473 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:38.320307 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448fa57-be3c-4292-b530-717df1ae2473 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:38.326084 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d242cc7-a647-48a4-974b-de0f61748106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:38.326499 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d242cc7-a647-48a4-974b-de0f61748106 admin admin] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:38.326738 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d242cc7-a647-48a4-974b-de0f61748106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:38.326930 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d242cc7-a647-48a4-974b-de0f61748106 admin admin] PATH_INFO: `/v3/projects` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:38.327736 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d242cc7-a647-48a4-974b-de0f61748106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:38.330048 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d242cc7-a647-48a4-974b-de0f61748106 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:38.392478 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1449/2884] 10.222.0.57 () {60 vars in 1152 bytes} [Thu May 26 21:17:38 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:17:38.396911 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b4a67-f9bc-43d4-8422-4a6084f6d8b0 None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:38.407671 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b4a67-f9bc-43d4-8422-4a6084f6d8b0 None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:38.476279 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab5dc9-6b43-4a7b-861d-6ee102256132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:38.476968 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab5dc9-6b43-4a7b-861d-6ee102256132 admin admin] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:38.477249 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab5dc9-6b43-4a7b-861d-6ee102256132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:38.477529 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab5dc9-6b43-4a7b-861d-6ee102256132 admin admin] PATH_INFO: `/v3/users` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:38.478247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab5dc9-6b43-4a7b-861d-6ee102256132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:38.480231 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab5dc9-6b43-4a7b-861d-6ee102256132 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:38.501879 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1436/2885] 10.222.0.57 () {60 vars in 1143 bytes} [Thu May 26 21:17:38 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:17:38.506625 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d242cc7-a647-48a4-974b-de0f61748106 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:38.517427 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d242cc7-a647-48a4-974b-de0f61748106 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:38.522740 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc5660-928c-4ae3-93d7-c732ee3b5556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:38.523425 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc5660-928c-4ae3-93d7-c732ee3b5556 admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:38.523711 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc5660-928c-4ae3-93d7-c732ee3b5556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:38.523993 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc5660-928c-4ae3-93d7-c732ee3b5556 admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:38.524722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc5660-928c-4ae3-93d7-c732ee3b5556 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:38.526339 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc5660-928c-4ae3-93d7-c732ee3b5556 admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:38.537444 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1450/2886] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:17:38.541794 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab5dc9-6b43-4a7b-861d-6ee102256132 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:38.552402 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab5dc9-6b43-4a7b-861d-6ee102256132 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:38.557183 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751cb5c-fbc2-4b09-a9c1-19f4cac68197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:38.558077 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751cb5c-fbc2-4b09-a9c1-19f4cac68197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:38.558367 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751cb5c-fbc2-4b09-a9c1-19f4cac68197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:38.558687 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751cb5c-fbc2-4b09-a9c1-19f4cac68197 admin admin] PATH_INFO: `/v3/projects/531db82a7d614586b53059ce8ff060ec/users/28c37030ce2947a591e6dddc4db1fc65/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:38.576080 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751cb5c-fbc2-4b09-a9c1-19f4cac68197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=531db82a7d614586b53059ce8ff060ec, user_id=28c37030ce2947a591e6dddc4db1fc65, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:38.578828 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6751cb5c-fbc2-4b09-a9c1-19f4cac68197 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:38.593342 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1437/2887] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:17:38 2022] PUT /identity/v3/projects/531db82a7d614586b53059ce8ff060ec/users/28c37030ce2947a591e6dddc4db1fc65/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:17:38.597331 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc5660-928c-4ae3-93d7-c732ee3b5556 admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:38.607718 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc5660-928c-4ae3-93d7-c732ee3b5556 admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:38.668925 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77527c2-453d-4989-909e-f47e319d73fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:38.669606 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77527c2-453d-4989-909e-f47e319d73fa admin admin] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:38.669884 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77527c2-453d-4989-909e-f47e319d73fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:38.670252 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77527c2-453d-4989-909e-f47e319d73fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:38.671014 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77527c2-453d-4989-909e-f47e319d73fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:38.672612 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77527c2-453d-4989-909e-f47e319d73fa admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:38.683027 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1451/2888] 10.222.0.57 () {58 vars in 1121 bytes} [Thu May 26 21:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:17:38.687358 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751cb5c-fbc2-4b09-a9c1-19f4cac68197 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:38.698847 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751cb5c-fbc2-4b09-a9c1-19f4cac68197 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:38.703652 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d5fc5-4eee-455a-aec8-b80e1fb199b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:38.704539 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d5fc5-4eee-455a-aec8-b80e1fb199b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:38.704823 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d5fc5-4eee-455a-aec8-b80e1fb199b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:38.705103 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d5fc5-4eee-455a-aec8-b80e1fb199b5 admin admin] PATH_INFO: `/v3/projects/531db82a7d614586b53059ce8ff060ec/users/28c37030ce2947a591e6dddc4db1fc65/roles/b8ccbdae4af04d82bdc9bbba141a6b8d` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:38.707174 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d5fc5-4eee-455a-aec8-b80e1fb199b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=531db82a7d614586b53059ce8ff060ec, user_id=28c37030ce2947a591e6dddc4db1fc65, role_id=b8ccbdae4af04d82bdc9bbba141a6b8d)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:38.710736 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d5fc5-4eee-455a-aec8-b80e1fb199b5 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:38.722151 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1438/2889] 10.222.0.57 () {60 vars in 1482 bytes} [Thu May 26 21:17:38 2022] PUT /identity/v3/projects/531db82a7d614586b53059ce8ff060ec/users/28c37030ce2947a591e6dddc4db1fc65/roles/b8ccbdae4af04d82bdc9bbba141a6b8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:17:42.973977 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce78803-1849-48d7-bf9e-6a6d8f08da0a None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:42.974657 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce78803-1849-48d7-bf9e-6a6d8f08da0a None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:42.974771 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce78803-1849-48d7-bf9e-6a6d8f08da0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:43.004139 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-bce78803-1849-48d7-bf9e-6a6d8f08da0a None None] MFA Rules not processed for user `28c37030ce2947a591e6dddc4db1fc65`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:17:43.005899 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-bce78803-1849-48d7-bf9e-6a6d8f08da0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:43.057749 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:43.079985 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1452/2890] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:17:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:17:43.096348 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d5fc5-4eee-455a-aec8-b80e1fb199b5 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:43.098111 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-0b4d5fc5-4eee-455a-aec8-b80e1fb199b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:43.176761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d5fc5-4eee-455a-aec8-b80e1fb199b5 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:43.182238 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cb4da-6830-44fe-9c57-577163c30cd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:43.182924 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb4da-6830-44fe-9c57-577163c30cd1 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:43.183211 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb4da-6830-44fe-9c57-577163c30cd1 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:43.183493 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb4da-6830-44fe-9c57-577163c30cd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:43.196746 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:43.244162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cb4da-6830-44fe-9c57-577163c30cd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:43.245845 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cb4da-6830-44fe-9c57-577163c30cd1 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:43.255053 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1439/2891] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:17:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:17:53.076960 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=5 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:53.250851 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=21 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:58.441370 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec250a-2adb-40d7-8464-63498282214c None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:58.441886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec250a-2adb-40d7-8464-63498282214c None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:58.441886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec250a-2adb-40d7-8464-63498282214c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:58.459867 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-6cec250a-2adb-40d7-8464-63498282214c None None] MFA Rules not processed for user `847b10367248482a8763e6b5d42f332c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:17:58.461110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6cec250a-2adb-40d7-8464-63498282214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:58.468067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1453/2892] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:17:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:18:08.454148 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=3 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:13.380551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cb4da-6830-44fe-9c57-577163c30cd1 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:13.381956 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-ea8cb4da-6830-44fe-9c57-577163c30cd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:13.437831 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cb4da-6830-44fe-9c57-577163c30cd1 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:13.441319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69e4c4-eeb2-4987-86cd-6d9765f26301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:13.441951 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69e4c4-eeb2-4987-86cd-6d9765f26301 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:13.442233 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69e4c4-eeb2-4987-86cd-6d9765f26301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:13.442542 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69e4c4-eeb2-4987-86cd-6d9765f26301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:13.451921 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69e4c4-eeb2-4987-86cd-6d9765f26301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:13.453369 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69e4c4-eeb2-4987-86cd-6d9765f26301 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:13.462705 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1440/2893] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:18:13 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:23.458506 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=26 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:33.385717 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec250a-2adb-40d7-8464-63498282214c None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:33.387135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-6cec250a-2adb-40d7-8464-63498282214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:33.455275 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec250a-2adb-40d7-8464-63498282214c None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:33.460273 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67e4f4-c804-491f-ac8d-b96228823c2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:33.460921 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67e4f4-c804-491f-ac8d-b96228823c2a service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:33.461232 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67e4f4-c804-491f-ac8d-b96228823c2a service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:33.461496 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67e4f4-c804-491f-ac8d-b96228823c2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:33.463284 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-cf67e4f4-c804-491f-ac8d-b96228823c2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:33.474592 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67e4f4-c804-491f-ac8d-b96228823c2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:33.476163 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67e4f4-c804-491f-ac8d-b96228823c2a service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:33.486578 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1454/2894] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:18:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:43.482228 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=32 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:20:15.242034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9d3c-838b-4782-940d-06ca4038935f None None] REQUEST_METHOD: `POST` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:15.242551 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9d3c-838b-4782-940d-06ca4038935f None None] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:15.242660 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9d3c-838b-4782-940d-06ca4038935f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:15.259568 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.auth.core [None req-fb7d9d3c-838b-4782-940d-06ca4038935f None None] MFA Rules not processed for user `847b10367248482a8763e6b5d42f332c`. Rule list: `[]` (Enabled: `True`). {{(pid=80269) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:15.260751 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-fb7d9d3c-838b-4782-940d-06ca4038935f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:15.267857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1441/2895] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:20:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:25.254624 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=3 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:20:31.882157 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67e4f4-c804-491f-ac8d-b96228823c2a service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:31.895083 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67e4f4-c804-491f-ac8d-b96228823c2a service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:31.900788 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e61bd-4d19-4718-b1c0-b0b15d0032cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:31.902067 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e61bd-4d19-4718-b1c0-b0b15d0032cd service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:31.902354 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e61bd-4d19-4718-b1c0-b0b15d0032cd service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:31.902666 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e61bd-4d19-4718-b1c0-b0b15d0032cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:31.904442 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-3f0e61bd-4d19-4718-b1c0-b0b15d0032cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:31.916682 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e61bd-4d19-4718-b1c0-b0b15d0032cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:31.918266 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e61bd-4d19-4718-b1c0-b0b15d0032cd service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:31.928843 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1455/2896] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:20:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:41.924225 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:20:42.042939 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d9d3c-838b-4782-940d-06ca4038935f None None] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:42.052545 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d9d3c-838b-4782-940d-06ca4038935f None None] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:42.056027 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21c892-7b91-413f-a874-b91b23141caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:42.056657 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21c892-7b91-413f-a874-b91b23141caf service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:42.056934 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21c892-7b91-413f-a874-b91b23141caf service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:42.057207 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21c892-7b91-413f-a874-b91b23141caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:42.059472 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7a21c892-7b91-413f-a874-b91b23141caf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:42.070133 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21c892-7b91-413f-a874-b91b23141caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:42.071613 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21c892-7b91-413f-a874-b91b23141caf service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:42.081435 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1442/2897] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:20:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:52.076825 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:22:04.444673 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f7111-7316-4e5b-aee6-d001672ce0b3 None None] REQUEST_METHOD: `POST` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:22:04.445193 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f7111-7316-4e5b-aee6-d001672ce0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:22:04.445353 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f7111-7316-4e5b-aee6-d001672ce0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:22:04.463292 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.auth.core [None req-f20f7111-7316-4e5b-aee6-d001672ce0b3 None None] MFA Rules not processed for user `847b10367248482a8763e6b5d42f332c`. Rule list: `[]` (Enabled: `True`). {{(pid=80268) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:22:04.464495 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-f20f7111-7316-4e5b-aee6-d001672ce0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:22:04.471377 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1456/2898] 10.222.0.57 () {58 vars in 957 bytes} [Thu May 26 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:22:04.513699 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21c892-7b91-413f-a874-b91b23141caf service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:22:04.515106 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-7a21c892-7b91-413f-a874-b91b23141caf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:22:04.589972 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21c892-7b91-413f-a874-b91b23141caf service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:22:04.593433 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5463ff-5f96-4517-b049-26f1fb9dce6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afe351334485400a8c11f17492c9ff5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:22:04.594060 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5463ff-5f96-4517-b049-26f1fb9dce6f service cinder] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:22:04.594339 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5463ff-5f96-4517-b049-26f1fb9dce6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:22:04.594639 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5463ff-5f96-4517-b049-26f1fb9dce6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:22:04.596373 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2a5463ff-5f96-4517-b049-26f1fb9dce6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:22:04.609256 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5463ff-5f96-4517-b049-26f1fb9dce6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:22:04.610711 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5463ff-5f96-4517-b049-26f1fb9dce6f service cinder] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:22:04.621586 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1443/2899] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:22:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:22:04.901768 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f7111-7316-4e5b-aee6-d001672ce0b3 None None] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:22:04.911402 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f7111-7316-4e5b-aee6-d001672ce0b3 None None] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:22:04.914947 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58978a8-ac8b-458b-8d2f-5b76438c0e62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:22:04.915580 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58978a8-ac8b-458b-8d2f-5b76438c0e62 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:22:04.915863 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58978a8-ac8b-458b-8d2f-5b76438c0e62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:22:04.916138 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58978a8-ac8b-458b-8d2f-5b76438c0e62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:22:04.917886 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-c58978a8-ac8b-458b-8d2f-5b76438c0e62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:22:04.928053 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58978a8-ac8b-458b-8d2f-5b76438c0e62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:22:04.929455 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58978a8-ac8b-458b-8d2f-5b76438c0e62 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:22:04.937865 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1457/2900] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:22:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:22:05.359857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5463ff-5f96-4517-b049-26f1fb9dce6f service cinder] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:22:05.361555 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-2a5463ff-5f96-4517-b049-26f1fb9dce6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:22:05.375958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5463ff-5f96-4517-b049-26f1fb9dce6f service cinder] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:22:05.377135 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58978a8-ac8b-458b-8d2f-5b76438c0e62 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:22:05.381093 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf07f4-a6c8-4ebb-8a10-d784cfbad049 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:22:05.381738 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf07f4-a6c8-4ebb-8a10-d784cfbad049 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:22:05.382031 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf07f4-a6c8-4ebb-8a10-d784cfbad049 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:22:05.382312 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf07f4-a6c8-4ebb-8a10-d784cfbad049 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:22:05.384609 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-4bbf07f4-a6c8-4ebb-8a10-d784cfbad049 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:22:05.385807 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58978a8-ac8b-458b-8d2f-5b76438c0e62 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:22:05.389471 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4bc1c-9185-4855-af9e-cdc2f9b948a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:22:05.390170 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4bc1c-9185-4855-af9e-cdc2f9b948a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:22:05.390560 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4bc1c-9185-4855-af9e-cdc2f9b948a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:22:05.390905 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4bc1c-9185-4855-af9e-cdc2f9b948a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:22:05.392798 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-a1c4bc1c-9185-4855-af9e-cdc2f9b948a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:22:05.397142 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf07f4-a6c8-4ebb-8a10-d784cfbad049 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:22:05.398760 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf07f4-a6c8-4ebb-8a10-d784cfbad049 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:22:05.407335 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4bc1c-9185-4855-af9e-cdc2f9b948a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:22:05.408785 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4bc1c-9185-4855-af9e-cdc2f9b948a4 service neutron] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:22:05.409723 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1444/2901] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:22:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:22:05.420605 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1458/2902] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:22:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:22:15.404996 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=41 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:22:15.414929 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=15 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:22:16.859824 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf07f4-a6c8-4ebb-8a10-d784cfbad049 service placement] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:22:16.875327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf07f4-a6c8-4ebb-8a10-d784cfbad049 service placement] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:22:16.880720 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e9580-16a0-4910-baa9-b0d3f82c5238 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:22:16.881365 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e9580-16a0-4910-baa9-b0d3f82c5238 service nova] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:22:16.881650 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e9580-16a0-4910-baa9-b0d3f82c5238 service nova] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:22:16.881927 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e9580-16a0-4910-baa9-b0d3f82c5238 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:22:16.883698 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-5c4e9580-16a0-4910-baa9-b0d3f82c5238 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:22:16.896327 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e9580-16a0-4910-baa9-b0d3f82c5238 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:22:16.897909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e9580-16a0-4910-baa9-b0d3f82c5238 service nova] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:22:16.909247 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1445/2903] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:22:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:22:26.904535 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:23:41.330723 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4bc1c-9185-4855-af9e-cdc2f9b948a4 service neutron] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:41.341687 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4bc1c-9185-4855-af9e-cdc2f9b948a4 service neutron] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:41.347338 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ee613-1bfe-4daf-a32c-26f3b9f82089 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:41.347968 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ee613-1bfe-4daf-a32c-26f3b9f82089 service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:41.348251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ee613-1bfe-4daf-a32c-26f3b9f82089 service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:41.348544 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ee613-1bfe-4daf-a32c-26f3b9f82089 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:41.356862 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ee613-1bfe-4daf-a32c-26f3b9f82089 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:41.358413 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ee613-1bfe-4daf-a32c-26f3b9f82089 service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:41.368421 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1459/2904] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:23:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:23:51.364255 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:24:04.013311 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e9580-16a0-4910-baa9-b0d3f82c5238 service nova] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:04.023199 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e9580-16a0-4910-baa9-b0d3f82c5238 service nova] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:04.026857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9ecd6-8f78-4c87-9497-ddb5aadee679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:04.027492 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9ecd6-8f78-4c87-9497-ddb5aadee679 service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:04.027776 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9ecd6-8f78-4c87-9497-ddb5aadee679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:04.028050 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9ecd6-8f78-4c87-9497-ddb5aadee679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:04.036332 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9ecd6-8f78-4c87-9497-ddb5aadee679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:04.037770 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c9ecd6-8f78-4c87-9497-ddb5aadee679 service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:04.046518 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1446/2905] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:24:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:24:07.189278 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ee613-1bfe-4daf-a32c-26f3b9f82089 service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:07.190722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-847ee613-1bfe-4daf-a32c-26f3b9f82089 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:24:07.206379 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ee613-1bfe-4daf-a32c-26f3b9f82089 service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:07.212046 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcaade-1320-497a-8d30-57ce65b7229d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '490d773bf21446789b77c5793a518a5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:07.212700 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcaade-1320-497a-8d30-57ce65b7229d service nova] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:07.212969 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcaade-1320-497a-8d30-57ce65b7229d service nova] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:07.213250 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcaade-1320-497a-8d30-57ce65b7229d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:07.215063 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-ffbcaade-1320-497a-8d30-57ce65b7229d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:24:07.229775 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcaade-1320-497a-8d30-57ce65b7229d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:07.231515 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcaade-1320-497a-8d30-57ce65b7229d service nova] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:07.243435 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1460/2906] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:24:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:24:14.042515 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=6 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:24:17.171991 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9ecd6-8f78-4c87-9497-ddb5aadee679 service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:17.181300 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c9ecd6-8f78-4c87-9497-ddb5aadee679 service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:17.184909 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89f9de-bb09-4a1d-80e9-b0a89e48522b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94b8d8b089264b5192a651b41bf2a205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:17.185605 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89f9de-bb09-4a1d-80e9-b0a89e48522b service neutron] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:17.185889 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89f9de-bb09-4a1d-80e9-b0a89e48522b service neutron] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:17.186164 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89f9de-bb09-4a1d-80e9-b0a89e48522b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:17.238627 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=6 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:24:17.244903 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89f9de-bb09-4a1d-80e9-b0a89e48522b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:17.246431 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89f9de-bb09-4a1d-80e9-b0a89e48522b service neutron] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:17.256168 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1447/2907] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:24:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:24:20.222803 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcaade-1320-497a-8d30-57ce65b7229d service nova] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:20.236044 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcaade-1320-497a-8d30-57ce65b7229d service nova] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:20.241116 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d68a4-6417-4b29-b4d3-ac489cfb658a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:20.241767 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d68a4-6417-4b29-b4d3-ac489cfb658a service placement] REQUEST_METHOD: `GET` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:20.242251 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d68a4-6417-4b29-b4d3-ac489cfb658a service placement] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:20.243011 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047d68a4-6417-4b29-b4d3-ac489cfb658a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:20.246315 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-047d68a4-6417-4b29-b4d3-ac489cfb658a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:24:20.262393 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d68a4-6417-4b29-b4d3-ac489cfb658a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:20.264381 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047d68a4-6417-4b29-b4d3-ac489cfb658a service placement] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:20.289191 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1461/2908] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:24:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:24:23.296560 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89f9de-bb09-4a1d-80e9-b0a89e48522b service neutron] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:23.308538 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89f9de-bb09-4a1d-80e9-b0a89e48522b service neutron] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:23.312857 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:23.313814 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:23.314196 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:23.314595 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] PATH_INFO: `/v3/users/847b10367248482a8763e6b5d42f332c` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:23.315888 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=847b10367248482a8763e6b5d42f332c)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:23.317958 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:23.371837 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '847b10367248482a8763e6b5d42f332c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.372930 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '847b10367248482a8763e6b5d42f332c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.380682 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '847b10367248482a8763e6b5d42f332c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.381319 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '847b10367248482a8763e6b5d42f332c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.394852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '847b10367248482a8763e6b5d42f332c'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.408034 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1448/2909] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:24:23 2022] DELETE /identity/v3/users/847b10367248482a8763e6b5d42f332c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:24:23.580190 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d68a4-6417-4b29-b4d3-ac489cfb658a service placement] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:23.591110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047d68a4-6417-4b29-b4d3-ac489cfb658a service placement] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:23.648247 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:23.649110 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:23.649391 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:23.649668 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] PATH_INFO: `/v3/projects/1b6678b588cc47f48b84d3120deccdb4` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:23.650846 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b6678b588cc47f48b84d3120deccdb4)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:23.652760 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:23.696825 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b6678b588cc47f48b84d3120deccdb4', 'request_id': 'req-8c86967e-b09c-43c0-926b-77853a6ef36d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.702722 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b6678b588cc47f48b84d3120deccdb4', 'request_id': 'req-8c86967e-b09c-43c0-926b-77853a6ef36d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.708797 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b6678b588cc47f48b84d3120deccdb4', 'request_id': 'req-8c86967e-b09c-43c0-926b-77853a6ef36d'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.720816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] The token cache is being invalidate because project 1b6678b588cc47f48b84d3120deccdb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:24:23.721117 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.721762 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.723369 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1462/2910] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:24:23 2022] DELETE /identity/v3/projects/1b6678b588cc47f48b84d3120deccdb4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:24:23.727332 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:23.728761 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:24:23.742325 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84e5b2-4470-413e-8dc8-04efcc0bca70 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:23.798606 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:23.799508 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:23.804028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:23.804028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] PATH_INFO: `/v3/users/28c37030ce2947a591e6dddc4db1fc65` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:23.804028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c37030ce2947a591e6dddc4db1fc65)` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:23.804028 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:23.810813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:24:23.852051 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c37030ce2947a591e6dddc4db1fc65'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.852715 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c37030ce2947a591e6dddc4db1fc65'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.859700 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c37030ce2947a591e6dddc4db1fc65'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.860363 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c37030ce2947a591e6dddc4db1fc65'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.871977 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.notifications [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c37030ce2947a591e6dddc4db1fc65'} {{(pid=80269) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:23.884277 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1449/2911] 10.222.0.57 () {58 vars in 1223 bytes} [Thu May 26 21:24:23 2022] DELETE /identity/v3/users/28c37030ce2947a591e6dddc4db1fc65 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:24:24.053231 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] Authenticating user token {{(pid=80268) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:24.054793 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.fernet_utils [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80268) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:24:24.068563 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86967e-b09c-43c0-926b-77853a6ef36d admin admin] Validating token access rules against request {{(pid=80268) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:24.125009 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c2ba123046b04572883d482380082522', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '116e4cb137e24acda75f964e89e322c5', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80268) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:24.126590 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:24.126876 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:24.127151 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] PATH_INFO: `/v3/projects/531db82a7d614586b53059ce8ff060ec` {{(pid=80268) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:24.128310 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=531db82a7d614586b53059ce8ff060ec)` {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:24.130209 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] RBAC: Authorization granted {{(pid=80268) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:24.150080 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:24:24.175894 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '531db82a7d614586b53059ce8ff060ec', 'request_id': 'req-f73cc379-6df8-40ff-8308-4d48c71fabcc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:24.181794 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '531db82a7d614586b53059ce8ff060ec', 'request_id': 'req-f73cc379-6df8-40ff-8308-4d48c71fabcc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:24.187178 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '531db82a7d614586b53059ce8ff060ec', 'request_id': 'req-f73cc379-6df8-40ff-8308-4d48c71fabcc'} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:24.199532 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] The token cache is being invalidate because project 531db82a7d614586b53059ce8ff060ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80268) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:24:24.199816 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:24.200423 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG keystone.notifications [None req-f73cc379-6df8-40ff-8308-4d48c71fabcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80268) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:24:24.201933 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: [pid: 80268|app: 0|req: 1463/2912] 10.222.0.57 () {58 vars in 1232 bytes} [Thu May 26 21:24:24 2022] DELETE /identity/v3/projects/531db82a7d614586b53059ce8ff060ec => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:24:33.880813 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:24:34.196592 dv-cinder-828806-3-iscsi devstack@keystone.service[80268]: DEBUG dbcounter [-] [80268] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80268) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:47.352961 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] Authenticating user token {{(pid=80269) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:47.354384 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:47.492127 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37050848-858b-4dde-9fe3-be237655fbb6 admin admin] Validating token access rules against request {{(pid=80269) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:47.548216 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086fb1cb-ffe2-474a-ace7-704ebb7b6561 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4edb53dc871047ce89ebb3b90143a6de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ebd2318798e41a3b42eacbc8fb55499', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80269) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:47.548852 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fb1cb-ffe2-474a-ace7-704ebb7b6561 service placement] REQUEST_METHOD: `GET` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:47.549134 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fb1cb-ffe2-474a-ace7-704ebb7b6561 service placement] SCRIPT_NAME: `/identity` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:47.549410 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086fb1cb-ffe2-474a-ace7-704ebb7b6561 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80269) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:47.551213 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.fernet_utils [None req-086fb1cb-ffe2-474a-ace7-704ebb7b6561 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80269) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:47.562058 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086fb1cb-ffe2-474a-ace7-704ebb7b6561 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:47.563673 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086fb1cb-ffe2-474a-ace7-704ebb7b6561 service placement] RBAC: Authorization granted {{(pid=80269) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:47.573335 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: [pid: 80269|app: 0|req: 1450/2913] 10.222.0.57 () {60 vars in 1365 bytes} [Thu May 26 21:28:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:28:57.569162 dv-cinder-828806-3-iscsi devstack@keystone.service[80269]: DEBUG dbcounter [-] [80269] Writing DB stats keystone:SELECT=89 {{(pid=80269) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}